Apple Watch Ultra 2 (vs Series 9) – Scientific Review

The Ultra 2 is great, but not better…

I didn’t think I would say this, but you might want to think twice about getting the Apple Watch Ultra 2. Or maybe a better wording is that I am personally conflicted about this watch. I have collected a lot of data with the Apple Watch Ultra 2 over the last 10 days, and what bothers me is that it doesn’t seem to be any better, and might even be worse in some cases, than the Series 9, even at things that the Ultra 2 should theoretically be better at. But you can judge yourself based on the data I will show you in this video, and there is plenty of data to share. Specifically, we will be testing the heart rate tracking accuracy of the Ultra 2 on myself and Theresa for a total of 42 workouts, and I’ll test the sleep stage tracking performance with 5 nights of data, we’ll be looking at the GPS tracking consistency using 18 bike-rides and we’ll check the oxygen saturation measurements using 100+ datapoints. I’ve put the Ultra 2 through some rigorous testing, and the results are super interesting. And to see if a big watch like the Ultra 2 also works on smaller wrists, I tested it on Theresa as well.

Check out my YouTube video:

Apple Watch Ultra 2 Improvements

However, first some quick basics about the Ultra 2. How is the Ultra 2 different from the Ultra 1 and the Apple Watch Series 9. Well, let’s first look at the upgrades compared to the previous generation. First of all, the screen is significantly brighter, now putting out a max brightness of 3,000 nits compared to the 2000 nits of the Ultra Generation 1. Now, even though this is noticeable in some situations when you hold them side-by-side, this probably will only occasionally make a significant difference in practice, but it is nice to have. Here you can see some footage side-by-side, and you can judge if this matters to you. And, as was highlighted during the launch event, the Apple Watch Ultra 2 now has the new S9 chip, which enables the new double tap feature. Now, for those of you that haven’t heard of double tap yet, it is basically a way of controlling basic Apple Watch functionalities by pinching your fingers twice. I haven’t been able to test this, since it hasn’t been released yet, but people like DC Rainmaker have been able to test it, and they say it works quite well. Finally, the Ultra 2 has Apple’s second-generation ultra-wideband chip, which means you can now very precisely locate your lost iPhone using Precision Finding, giving the exact distance to the phone. This is actually a valuable feature that I would use, since I often misplace things, and I already use my AirTags for this quite a lot, so this is a nice addition. When it comes to voice-inputs there are also some upgrades: first of all Siri‌ can now process some commands on-device for requests that do not need information from the internet, which results in quicker and more reliable responses; and Siri will be able to access and add health and fitness data, but this feature still needs to be released. The dictation itself is also supposed to be about 25% more accurate. Those are the most important changes compared to the generation 1 of the Ultra, which are honestly quite minor changes. I think it is more interesting to look at the differences between the Ultra 2 and the Apple Watch Series 9. Now, similar to when we compared the Ultra 1 to the Series 8, the Ultra 2 has a much longer battery life than the Series 9. The Ultra 2 has a battery life of up to 36 hours, whereas the Series 9 has a battery life of up to 18 hours. This is for me the main benefit of the Ultra and Ultra 2; I can forget to charge my smartwatch for one day, and I will still be fine. Luckily, both the Ultra 2 and Series 9 have fast-charging, with the Ultra 2 taking about 1 hour to charge to 80%. What is far more important is the precision dual-frequency GPS that the Ultra 2 has, which should be better than that of the Series 9. However, as you will see later in this review, there are some weird things going on here. Finally, as you know, the Ultra series is geared much more to the adventurous outdoor types, so there are all sorts of functionalities that make the Ultra 2 especially good for hiking, diving, marathons and more. However, you can easily find out more about this on Apple’s website, so I won’t discuss that. Instead let’s get to the thing you’ve all been waiting for: the systematic testing of the Apple Watch Ultra 2, and as always I want to start off with the heart rate tracking performance of the Ultra 2, which I tested on myself and on Theresa for a total of 7 indoor cycling sessions, 21 outdoor bike rides, 7 weight lifting sessions and 7 outdoor runs.

In case you are interested in all heart rate results, these can be found here:

Now, to test the performance, I’ll compare the heart rate measurements of the Apple Watch Ultra 2 against the Polar H10 ECG chest strap, which can generally record my heart rate very accurately. And we will start by looking at one of the easiest types of exercise for a watch to track: cycling indoors. Here we see an overview of that accuracy for the Apple Watch Ultra 2 on me. Each dot here is a single heart-rate measurement, with on the horizontal axis the value according to the Polar H10 chest strap, and on the vertical axis the value according to the Apple Watch Ultra 2, and we want the dots to be as close to the blue line as possible.

Indoor Cycling Test 1 : Male

Here we see the interval spinning session, with along the horizontal axis we have the time, and my heart rate is along the vertical axis. In blue-green I plotted my heart-rate according to the Polar H10 chest strap, and in red is my heart rate according to the Ultra 2.

Indoor Cycling Test 2 : Female

Indoor Cycling Test : Comparison w/ Other Watches

That’s looking really good, however, how does this compare to the Series 9 and other smartwatches? Well, let’s put these results into context and let’s compare the Ultra 2 against almost 70 different watches I’ve previously tested. This way, you can determine if the Ultra 2 is the watch for you!

That overview is displayed here. The correlation value I was talking about before is the metric I will use for this, which is displayed along the horizontal axis here. We want that value to be as close to 1 as possible. On the vertical axis I ordered the watches from worst to best, so the further to the right and the higher a device is, the better its correlation with the reference device. And I marked the Apple Watch Ultra 2 in red.

Outdoor Cycling Test 1 : Male

Here are the sessions themselves:

Outdoor Cycling Test 2 : Female

Outdoor Cycling Test : Comparison w/ Other Watches

Running Test 1 : Male

Here are the sessions:

Running Test 2 : Female

Weight Lifting Test 1 : Male

Weight Lifting Test 2 : Female

Here are the sessions themselves:

Weight Lifting Test : Comparison w/ Other Watches

Heart Rate Conclusions

Overall, the Apple Watch Ultra 2 still has an amazing heart rate tracker. However, it seems to perform ever so slightly worse than the Series 9. This makes sense, as the added weight of the Ultra 2 likely increases movement of the watch on the wrist during certain exercises. Still, the differences are relatively minor, and Apple has generally developed the best heart rate sensor on the wrist, at least out of all watches that I’ve tested. So, the Apple Watch Ultra 2 is still an amazing heart rate tracker. Just keep in mind there might be some small performance decrease compared to the Series 9. Therefore, since the Apple Watch Sensor is still the best performer out there, with potentially just a minor decrease in performance due to the weight of the Ultra 2, I would give the heart rate tracking, 4 and 3/4 out of 5 stars, a quarter less than I gave the Series 9.

Sleep Test : Ultra 2

Let’s now dive into a feature that is extremely important to me: sleep stage tracking. The apple watch tracks your 4 major sleep stages, light sleep, deep sleep, REM sleep and awake time, if we count the latter as a sleep stage, though as you know, Apple calls light sleep “core sleep” instead. I will compare the sleep stages as tracked by the Ultra 2 against the Z-Max EEG device, which can measure my brainwaves. Now, I won’t go into details, since I already discussed this extensively in my Series 9 video, but the Z-max likely overestimates awake moments, so I will just focus on deep sleep, REM sleep and light sleep in this review.

First the confusion matrix:

Now the data for all nights (too large to display all). You can download them here:

Sleep Test : Ultra 2 vs Series 9

Here is a side-by-side comparison:

Sleep Test : Comparison w/ Other Watches

This graph shows an overview of the agreement of different watches with the EEG device. On the horizontal axis we have the average agreement over the individual sleep stages, and on the vertical axis we have the agreement of the worst sleep stage. The better the agreement, the more to the top right a device is. Now, this overview is slightly complicated because I used different reference devices.

The sleep stage tracking of the Apple Watch Ultra 2 is really good, and Apple really is the market leader when it comes to sleep stage tracking. Therefore, overall, I’d give the sleep stage tracking performance of the Apple Watch Ultra 2, 4.5 out of 5 stars. It doesn’t get 5 stars, since even the Apple Watch can still improve a bit.

Oxygen Saturation Test

Next, let’s discuss a feature of the Ultra 2’s that potentially helps detect sleep disorders through oxygen saturation measurements. This could be helpful in identifying conditions like sleep apnea, where breathing pauses cause reduced oxygen levels. Ideally, I’d test the Ultra 2 in a low oxygen environment, but that’s possible only in two weeks or so when I return to the Netherlands. Meanwhile, I’ll check if it wrongly detects low oxygen levels in a normal environment by taking many measurements.

Limitations

So luckily the Apple Watch Ultra 2’s blood oxygen saturation measurements were generally within the normal range, with just a few quite low measurements. However, my final judgment will have to wait until my tests in low oxygen environments are complete. Now, before giving you my final conclusions, there are definitely limitations to my testing, which include the small sample size and potential inaccuracies of reference devices. Despite these, based on my results and reviews by other YouTubers like DC Rainmaker, DesFit, Chase The Summit and Matt Legrand, Apple Watches are typically reliable heart rate trackers- although I do recommend you check their reviews too. Also, bear in mind that firmware updates could alter results in future.

Recommendations, Wish List & Conclusions

Still, the Apple Watch Ultra 2 is an amazing heart rate tracker, the sleep tracking is also the best out there, and the GPS tracking is pretty good, though I am surprised it isn’t better. When it comes to oxygen saturation more testing is needed, though it generally performs as expected. Therefore, I’d still rank the Ultra 2 among the best for sports and health tracking, scoring it between 4 to 4.5 out of 5 stars. If you want to see more of the data that I didn’t have time to show in this video, I will upload it to my website which is linked below. All of this being said, even though the Apple Watch Ultra 2 is generally amazingly accurate, I do have a few complaints: first of all, even though the battery life of the Ultra 2 is better than that of the Series 9, it is still quite bad compared to the competition. I do wonder if there is really no way that Apple can extend this a bit more. Second, even though it bears the “Ultra” name, I don’t think the Ultra 2 has shown to be more accurate than the Series 9 in my testing, which is disappointing. Third, even though the health app and especially the fitness app look really beautiful, I think they just aren’t super functional. I would like to get all my health stats in a single app with clear explanations and advice. This is where the Oura Ring and Whoop Strap do much better, and these are still my go-to devices when it comes to my daily health tracking. Whoop actually recently introduced a personalized chatGPT based AI coach that you can ask for advice, which it will provide based on your own data. I think this is super cool, and this is likely where things are headed, with personalized actionable advice that is put into context and explained. I still need to test this feature more, but I hope Apple will also provide something like this in the future. But, all of this being said, who is the Apple Watch Ultra 2 for? Well, the same people that the Ultra 1 was for: people who need a rugged watch with extra functionalities for more extreme outdoor sports activities. You also get extended battery life and a bigger screen. However, if you already own an Ultra 1 there is really no reason to upgrade, and if you don’t mind charging your watch every day, the Series 9 will likely suffice for most people. And, honestly, the Apple Watch SE is probably good enough too, which will save you a ton of money. Now, if your focus is general health tracking with some additional sports tracking, the Apple Watch would not be my first choice, not because the data is not good enough, because Apple Watches are generally the most accurate, but just because the data presentation of Apple just isn’t the best at the moment. For this I recommend the Oura Ring for just health tracking and the Whoop strap if I also want to track your sport activities. If you want the best GPS tracking, Garmin has got you covered, and if your budget is a bit more tight, Coros would be a cheaper alternative. Finally, if you want great sleep tracking and additionally want to actively improve your sleep quality, I would get an Eight Sleep Pod 3.