The new PowerBeats 2 Pro have a slate of new features for audio-related purposes. However, for sports-specific usage, they have one big new feature: Built-in heart rate sensing. This would not only be the first time Apple has created a non-watch heart rate sensor, but even more notably, the first time the company has enabled heart rate broadcasting over existing Bluetooth heart rate standards.
The question then becomes: Is it accurate?
Unfortunately, it turns out, that was not the question I should have started with.
The real question to start with is: Is the heart rate function (accuracy aside), even usable?
A lot of hours later, I have answers to both of those questions. And trust me, it’s a very mixed bag.
I’ve actually been a long-time fan of PowerBeats. There were the first Beats product I owned years ago, before eventually switching to the Beats Studio buds, then the AirPods Pro 2. But hey, I’m a fan of orange (as is every other company apparently), so a quick $249USD (299EUR in my case) purchase on Apple.com, and they arrived two days later.
Note that this review will focus exclusively on the heart rate features. This is not a review of the audio pieces (good or bad), the other new features, or the color orange. Cool? Good, let’s begin.
Finally, if you found this review useful, consider becoming a DCR Supporter, which gets you an ad-free site, plus the behind-the-scenes video series between both myself (and my wife) on everything that happens in the DCR Sports Tech Cave/universe.
Heart Rate Pairing & Enablement:
Never have I ever spent so much time trying to get an Apple product to work. And actual the solution? Use Android.
I’m not kidding. After over 90 minutes of troubleshooting trying to get heart rate sensor pairing working within the Apple ecosystem, I gave up and turned to Android to capture my heart rate data for accuracy testing. It was the only way I could make it work. Still, I’ll cover how this all works, both in theory, and in practice.
First up, the PowerBeats 2 Pro are fully compatible with both Apple and Android. In the case of Apple, all of the various PowerBeats extra features are handled at the OS level, so you can toggle things on/off as needed (like noise cancellation, heart rate, etc…). Whereas on Android, you’d use the Beats app to do the same thing.
Now before we dive down that rabbit hole, let’s look at the tech. You’ll see a small black sensor pod on the inside, just below the eartips. This has both a green light (technically 3 green lights) and a photodiode to measure said lights:
It shines this light into your ear and then reads that optically, determining your heart rate. The green light is very visible to all of those around you:
In daylight, you don’t notice it very much. Though, in darker conditions it’s very visible:
This is a separate space from where the music comes out, so there isn’t (theoretically) and interference between these two sensors. For some tests I had music playing, and other tests not. Music playback didn’t appear to make any difference in terms of accuracy.
Now, transitioning to the heart rate software side, the PowerBeats 2 Pro supports two core standards-based protocols:
A) Bluetooth Audio device
B) Bluetooth Heart Rate profile
It does this on two distinct channels, theoretically allowing you to pair individually the heart rate side to one device (gym equipment), while the audio side stays connected to another device (like your phone, for music). Spoiler: This doesn’t actually work.
To pair the music side on an iPhone, you’ll open the case up with an Apple device nearby, and it’ll magically pop-up and start the pairing process. Clean and simple:
In the case of Apple, it’ll sync this pairing across all your Apple devices (e.g. Mac, iPhone, etc…).
Meanwhile, on an Android device, you’ll open the case lid and hold down the single button within it, to enable pairing mode. A moment or two later, it’ll show up in your Bluetooth devices and you can confirm the pairing. Again, also straight-forward and simple.
Now on both Android and iOS, you can entirely disable heart rate monitoring (which is only enabled during workouts, not 24×7/during music). So if you don’t want it to ever enable, you can toggle that easily.
Meanwhile, assuming you do care about that, then it’s enabled by default. However, it’s not measuring/broadcasting yet. To do that, you need quickly short press followed by a long press on either of the two Beats buttons (left or ride side). You’ll hear a chime, which then enables the heart rate sensor for pairing. At which point, you should be able to pair it to various apps/devices as a standard Bluetooth audio device.
In reality, this is a hot mess.
I tried numerous apps on iPhone and Mac: Peloton, Zwift, Strava, Wahoo Fitness. Plus, Garmin watches and bike computers. About half the time it’d see the heart rate sensor, but then the subsequent pairing would fail. It’d make a disconnecting sound in the headphones, and a short bit later the app/device would show a failed attempt:
I’ll save you a crapton of rebooting/updating/troubleshooting/powering on/off/etc… The solution, was to remove the Beats PowerBeats 2 Pro from my Apple account (since it syncs across multiple Apple devices), and try directly with Android. Oddly enough, this worked flawlessly. I was immediately able to get heart rate data on Android across numerous apps, without issue. Below, pairing within Strava (left), Wahoo Fitness app (middle), and mid-workout on Wahoo app (right):
You can see the pairing here on the Google Pixel (Android) phone, showing a heart rate of 171bpm from the PowerBeats 2 Pro (Zwift on the Mac, is paired to a Polar H10 chest strap):
Interestingly, in my troubleshooting, if I left the pairing with my iPhone, the moment I unlocked my iPhone, it’d steal both connections and break the heart rate connection to Android. Thus again, the only solution being to fully remove from Apple.
Less you think this was just me, some 5,000 miles away, Des of DesFit (another YouTube fitness channel, and my co-host in the Fit File Podcast), was concurrently troubleshooting the exact same process for equally as long. Likewise for Shervin Shares (YouTube Fitness reviewer), noted on Instagram he was unable to get HR to work and also had to fail back to Android. Said differently: If we can’t get this to work, y’all are hosed.
In discussions with Apple, it appears a substantial limitation is that the Beats cannot actually work with gym equipment for heart rate, and your phone playing music at the same time. A Beats PR/media representative has confirmed this, saying: “You cannot listen to music with your iPhone while also paired with the gym equipment for heart rate.”
Note, Apple & Beats emphasize the gym equipment heart rate monitoring in their marketing of the PowerBeats 2 Pro, as well as the first few screens when you setup the PowerBeats 2 Pro on your iPhone:
In any event, with it working on Android, off I went to record some data and start accuracy testing.
Heart Rate Accuracy:
For these accuracy tests, I’m comparing heart rate data to multiple other trusted heart rate sensors. This includes a heart rate chest strap, two other watches, and optical heart rate bands. In the case of the Apple Watch Ultra 2, this was on a separate Apple account, and more importantly, the PowerBeats 2 Pro were fully removed from my Apple account. Else, Apple will automatically leverage Apple Watch data over PowerBeats 2 Pro data (a silly automatic configuration btw).
Let’s start with the first workout, an indoor trainer ride. This ride had some nasty-hard intervals in it, but was a great test to see how things worked. Here it is versus a Polar H10 chest strap, an Apple Watch Ultra 2 optical HR sensor, and a Garmin Forerunner 965 optical HR sensor (Gen 4):
(Note: The purple FR965 track, I simply forgot to hit the start button, hence the delayed start there)
As you can see, at first glance it’s actually pretty darn good. And indeed, on the whole it was. However, there are two parts that are somewhat hidden in here where the heart rate dropped out entirely as I hit harder intervals. Because of the fact that the graphing software connects the dots, it appears to follow the rising slope of the other units, when in reality, it just connected two dots after upwards of a minute of no heart rate value displayed. You an see this here on the phone displaying the PowerBeats data, showing no value for Heart Rate (BPM):
Here’s how I’d rank the ride overall (reds are mostly dropouts):
Again, on the whole, if we set aside the higher-intensity dropouts, it’s not too bad actually.
Next, we’ve got an interval run with hill repeats. I start off with a nice warm-up, though, getting the PowerBeats 2 Pro to the start line was a struggle bus of waiting for it to find HR lock, then it would lose it, and then find it, and over and over for minutes. Once I finally found it…well…it lost it immediately. In any case, here’s that data set:
In fact, what we quickly realize is that Beats is very clearly prioritizing accurate data over inaccurate data. Which is to say, when it appears their confidence level is low, they cut-off all heart rate data entirely. Literally, transmitting nothing (no value). I could see the logic in that if it happened rarely, but in reality, it happens constantly.
Where we can see that. I tried various tweaks to fit during the first 10-12 minutes, and it didn’t seem to matter any. In fact, the only thing that seemed to matter was actually running harder. Perhaps it was sweat, perhaps warming up, either way, this is by any definition a hot mess.
So, then we look at the actual accuracy, and here’s how that handles (compared to the other HR sensors, which were all near identical, though, the Apple Watch Ultra 2 did struggle briefly on two intervals).
As you can see, it was not good.
Still, I gave things another whirl for one last run the next morning. Perhaps it just needed to sleep on it. I also ensured the firmware was fully up to date, and that…well…that’s it.
So, off I went, music inclusive this time (again on Android):
During the run, I could already tell things were gonna be bad. Dropouts were frequent, and when it made sense, I would stop and let the heart rate try and fix itself, aligning to the other straps (something I’d never really do). Note the double-dash on the middle row, indicating no HR values. Music continued to play without issue.
Otherwise, I just kept running. As you can see, things aren’t good:
Just to make this a bit more clear, here’s all the points the heart rate had dropped out:
And, even when it wasn’t dropped out, it was still bad-bad. While the year is still young, this is definitely in the running for the worst optical HR sensor in a long while.
The thing is, doing more tests is kinda pointless. It’s clear that any movement (even on a stationary bike) quickly leads to either dropouts or inaccurate heart rate. And outdoors running, it’s even worse. Ultimately, I don’t see any value in the heart rate sensor in this product, because it’s simply not good enough to be useful, even for casual use.
(Note: All of the charts in these accuracy sections were created using the DCR Analyzer tool. Though, I manually overlaid the color blocks on this one. It allows you to compare power meters/trainers, heart rate, cadence, speed/pace, GPS tracks, and plenty more. You can use it as well, more details here.)
Wrap-Up:
I had high hopes for the PowerBeats 2 Pro, mainly because Apple has traditionally done very well with optical heart rate sensing technologies. My assumption was that Apple wouldn’t have released a product with inaccurate data. And indeed, the company frequently touts how many hours of testing with athletes it’s done (stated at 1,500 hours of testing on 1,000 athletes):
However, I’d actually argue the biggest issue with the PowerBeats 2 Pro isn’t (somehow) the heart rate accuracy. It’s simple getting the heart rate sensor data into Apple devices or 3rd party devices. It’s darn-near impossible to do consistently (or even inconsistently), unless I use Android. I fail to understand how this got shipped, or how this passed any quality bar/tests. When it seems none of the fitness reviewers can make this work (and none of the mainstream tech reviewers covered it), that’s a very legit problem. The limitation of being unable to use the heart rate data on gym equipment and play music is downright bizarre.
Nonetheless, if you’re an Android user, and plan to use it wholly within the phone (meaning, playback of music on the same phone as recording heart rate), and if you do activities with minimal movement (not running outside, or Zumba), then, I guess I kinda works. For my indoor ride, accuracy was largely quite good, until I got to the tippy-top pain locker of some hard intervals where I was sprinting a bit (then it lost track of HR). Otherwise, it was fine. Of course, running it was a hot mess.
Looking at comparisons, ultimately, the story here isn’t actually much different than the Sennheiser Momentum Sport earbuds that came out a year ago. Except, I’d actually argue those sounded better, though, that had even worse heart rate performance. Like, it made even the PowerBeats 2 Pro run look picture-perfect. I only managed a single salvageable workout on those. Though to its credit, at least it properly paired without any issues whatsoever (including to an iPhone).
If I look back over the course of the last decade of companies touting ear-based heart rate monitoring/sensors, all of them keep saying that it’s “the best place” to monitor heart rate. Yet, over the last decade, all of them clearly demonstrate that it’s simply not. From tiny startups, to midsized companies, and of course the behemoth that is Apple themselves. Every product I’ve tested simply isn’t that good.2
In some ways, I know understand Apple’s somewhat frustrating decision that forces Apple Watch users data stream to the Apple Watch’s optical HR sensor over the PowerBeats 2 Pro: The Apple Watch HR sensor actually works. And thus, that team has our back and knows better.
With that – thanks for reading!
0 Commentaires