Summary

Last year, Apple released the fourth generation of the iPad Pro. Its built-in LiDAR sensor was a major upgrade of previous generations of the device. The sensor unlocked an array of new capabilities, ranging from 3D scanning to enhanced augmented reality (AR).

When the 2020 iPad Pro was released, vGIS tested it and its LiDAR sensor using various performance metrics. We concluded that, despite a few missed opportunities, LiDAR is beneficial for AR. Click on these links to read our reviews, published in April 2020 and December 2020:

2020 iPad Pro: Does the LiDAR sensor improve spatial tracking?

iPhone and iPad LiDAR spatial tracking capabilities: second test

This year, the major iPad Pro update was the new M1 Apple chip. Apple also boosted the maximum RAM for the iPad Pro to 16 gigabytes. The combination of improved CPU and expanded memory promised a CPU speed up to 50 percent faster than that of Apple’s 2020 chip, the A12Z Bionic, and a GPU speed up to 40 percent faster than that of the A12Z. The new chip means that the iPad Pro will be the undisputed heavyweight performance champion for years to come.

Augmented reality is one of the most resource-intensive processes that a phone or tablet can perform. In addition to real-time rendering, which is already very taxing, AR requires real-time video processing, further straining the resources of the device. So any system that can deliver boosted performance to accommodate AR is welcome, and the new iPad Pro caught our attention.

Intrigued by the possibilities, we set out to investigate the following questions:

• Is the excitement about performance gains warranted?
• Does it make sense to upgrade from the already powerful 2020 iPad Pro?

In this article, vGIS—the leading developer of augmented reality (AR) and mixed reality (MR) solutions for field services—answers these and other questions.

Introduction

Apple announced the 2021 iPad Pro in April 2021. The most exciting detail of the announcement was the transition to Apple’s M1 chip. The company’s press release stated that, according to Greg Joswiak, senior vice president of worldwide marketing, “The revolutionary M1 chip has been a breakthrough for the Mac, and we’re incredibly excited to bring it to iPad Pro.”

According to Apple, “The breakthrough M1 chip takes the industry-leading performance of iPad Pro to an entirely new level. The 8-core CPU design features the world’s fastest CPU cores in low-power silicon—delivering up to 50 percent faster CPU performance than A12Z Bionic. The 8-core GPU is in a class of its own, delivering up to 40 percent faster GPU performance. This combination of CPU and graphics performance on iPad Pro widens its lead as the fastest device of its kind. Powerful custom technologies, including a next-generation 16-core Apple Neural Engine, an advanced image signal processor (ISP), a unified, high-bandwidth memory architecture with up to 16GB of memory, 2x faster storage, and up to 2TB capacity, make iPad Pro more capable than ever.”

In augmented reality, performance matters. In the Android ecosystem, every new generation of Snapdragon CPUs—from 855 to 865 to 888—has brought noticeable improvements, including a smoother AR experience made possible by higher frame rates and shorter rendering times. Such improvements have not been equally conspicuous in every form of computing. When we upgraded the 2017 desktops that we use to develop vGIS to 2021 specs, we barely noticed any difference. But every new generation of mobile CPU and GPU has brought solid gains in performance.

In the world of high-performance vGIS augmented reality, the numbers presented in Apple’s announcement are spectacular. vGIS is designed to visualize BIM, GIS, and 3D scans (reality capture data) with high accuracy, and we often feel constrained by the available hardware. The transition to M1 chip and up to 16Gb of RAM is not a typical iterative upgrade from one CPU generation to the next. In terms of performance and capabilities, it is a leap forward.

Several benchmarking tests have confirmed the impressive performance of the iPad Pro. However, such tests do not always accurately reflect real-life results. So we also compared the performance of 2021 iPad Pro to that of the previous generation of iPad Pro, to the iPhone 12 Pro, and to a few Android devices. We wanted to know:

• Is the 2021 iPad Pro better equipped for AR than other devices?
• Is it worth upgrading from the 2020 iPad Pro to the 2021 iPad Pro?

Traditional AR relies mainly on processing live video feed captured by the cameras of a device. The technology is not perfect; the live video processing may fall prey to misinterpretations and unfavorable light conditions like glare, solid surfaces, or dim light and bright light. But LiDAR emits its own light and offers superior spatial tracking. Under normal conditions, LiDAR can identify distinct features much more clearly than purely optical tracking can. It then uses those features to anchor the augmented reality in space.

For good measure, we also checked LiDAR performance. In this test, we assessed the ability of devices to:

• Detect surfaces, like pavement.
• Track movements and correct for the positional drift typically associated with such tracking.

Methodology

Primary Test: Comparative iPad 2021 Performance

To assess device performance using conditions regularly experienced by vGIS users, we measured the processing time of BIM and GIS models of three levels of complexity:

• Light. Original GIS or BIM data source of 150 kilobytes.
• Medium. BIM model of about 40 megabytes.
• Heavy. Tiled 3D reality capture scan of 120 megabytes.

The second part of the test projected the medium-sized model while visually observing the rate of frames per second (FPS) to gauge the quality of the user experience under such a load.

Secondary Test: LiDAR-assisted AR performance

We made the comparison using the vGIS system without an external GNSS. This way, the devices had to rely entirely on AR for positioning.

Surface scanning is a key background process in vGIS, enabling critical functionality and improving the accuracy of AR visuals. We reviewed how well iPads and other devices performed when determining the elevations of the following surfaces:

• Glossy hardwood.
• Nearby surfaces; e.g., a tabletop about one foot or 30 centimeters from the device.
• A nonreflective floor with distinct features.
• A complex grass surface.
• Concrete with distinct features.
• Even pavement without distinct features.

In the second part of the test, we prepared a simple closed-loop course that was about 100 meters long, and we performed several runs with each device to measure deviation from the marker at the start position and the end position.

Devices

Our tests were conducted using the following devices:

• iPad Pro 12.9 (2021) with M1 CPU, 16 GB RAM, and Apple GPU (8-core graphics)
• iPad Pro 12.9 (2020) with A12Z Bionic CPU, 6 GB RAM, and Apple GPU (8-core graphics)
• iPhone 12 Pro with Apple A14 Bionic, 6 GB RAM, and Apple GPU (4-core graphics)
• Sony Xperia 1 II with Snapdragon 865, 12 GB RAM, and Adreno 650 GPU
• Samsung Galaxy Note 10+ with Snapdragon 855, 12 GB RAM, and Adreno 640 GPU
• LG V60 ThinQ 5G with Snapdragon 865, 8 GB RAM, and Adreno 650 GPU

Performance Test Results

Rendering times

The following are the average rendering times for each device based on ten rendering attempts. The numbers are in minute and seconds, rounded to the nearest second.

The videos of test runs are below.

Medium model

Heavy model

FPS with medium model

Performance Test Summary

With respect to rendering, the light model did not result in any noticeably different wait times.

The medium-sized model revealed a large gap between the performance of iOS and Android. Although iOS devices finished under 20 seconds, the Sony and LG devices took almost three times as long to finish, and Samsung took two minutes to finish. The gap was extremely pronounced and consistent.

Surprisingly, rendering the largest model took the 2020 iPad Pro almost as much time as it took Android devices; they all clustered in the range of two minutes to two minutes and 30 seconds. This time, Samsung was slightly ahead of Sony. The 2021 iPad Pro consistently took half as much time to finish rendering as the other devices. The iPhone 12 Pro was unable to finish rendering at all, since iOS kept terminating the app.

LiDAR and Spatial Tracking Results

Since the vGIS app is used outdoors, we also tested spatial tracking and surface detection.

The results of the surface detection test were almost the same as the results of our fall 2020 test: the iPad Pros, iPhone 12 Pro, and Samsung Galaxy Note 10+ performed admirably. The Sony Xperia 1 II did a very good job; but since it does not have LiDAR or useful time of flight (its ToF is not used for enhancing AR), it suffered from occasional glitches. The biggest surprise was the LG V60 ThinQ 5G. Although listed as having ToF-enabled AR depth sensing, its surface detection results were the worst by far.

Here are the results for positioning tracking accuracy and precision (smaller is better).

For an explanation of the difference between accuracy and precision, please see our previous test.

In this test, iOS devices worked as if they were on rails, producing consistent and accurate results time after time. Samsung also performed well, assisted by the superior drift correction of Android, but fell slightly behind iOS in precision. Sony did a surprisingly good job for a non-ToF-assisted device. By far, LG performed worst, accumulating significant drift over the hundred-meter walk.

Conclusions

Is 2021 iPad Pro the best choice for AR?

Absolutely.

When it comes to AR performance, iPhones and iPads beat their Android counterparts by a wide margin. Comparing high-end Android devices to high-end iOS devices, or mid-range Android devices to mid-range iOS devices, leads to the same quantifiable result: AR on iOS exhibits superior performance. The disparity in performance is especially pronounced in the case of mid-range loads, when the iPhone and the iPad finish rendering models 40 percent to 60 percent faster than Android devices. Doubtless impeded by its older GPU and bloatware, Samsung performed the worst. Although the iPads and the iPhone finished rendering the medium model within 20 seconds, the Samsung phone needed almost two minutes.

In our load handling test, the heavy model produced mixed results. iPad is a beast when it comes to processing large volumes of data; it is in a league of its own. However, other iOS devices were on a par with Android. The iPhone was unable to render large datasets at all, since the OS kept aborting the AR app to prevent runway consumption of memory.

iOS also offers smoother AR user experience, more consistent and accurate surface detection, better depth-mask detection, and superior spatial tracking without any significant degrading of performance. All things considered, Android devices are far from being on a par with iOS with respect to augmented reality. Google has a lot of catching up to do.

The ability of iOS to handle LiDAR is a major reason to choose iOS over Android. But not everything is rosy in the iOS garden. If you standardize around iOS, keep in mind that this operating system is very restrictive in its support for accessories. We find that professional hardware is often incompatible with iOS. So if you plan to connect GNSS devices or other external hardware to your iPad using Bluetooth, first check for compatibility.

Is it worth upgrading from iPad 2020?

It depends on what you are planning to do with the device. If you process extremely large data sets in AR on a regular basis, and you want to guarantee that your device can handle most of these data sets, you should upgrade to the 2021 iPad Pro. It is the king of performance. If the 2021 iPad Pro cannot handle it, no other mobile device can either.

Even for vGIS users, though, rendering massive datasets or massive graphics is only a small proportion of all use cases. For other applications, you probably won’t notice any difference between the 2020 iPad Pro and the 2021 iPad Pro. Unless you really do need the best to handle the most, the 2020 iPad Pro is a great choice. Under normal conditions, you won’t notice any difference in performance. Moreover, the 2021 iPad Pro reportedly limits the memory size to 5 gigabytes per application, so you won’t be able to take much advantage of the larger RAM.

Unexpected Findings

Our tests often lead to unexpected findings. Sometimes the findings are pleasantly surprising. Other times we’re left scratching our heads.

The 2021 iPad Pro: held back by iPadOS

The 2021 iPad Pro is an impressively powerful mobile device that offers the best AR experience. But it is held back by lackluster software. iPadOS 14 and iPad OS 15 feel both polished and underwhelming, like a high-tech sledgehammer. Even if it’s made of the strongest alloy, has the best shape for the job, and has the most comfortable and rugged handle—it’s still just a sledgehammer. It doesn’t delight or surprise. Neither does iPadOS (unlike Android OS).

Bent iPad

After taking the 2020 iPad Pro out of its case a couple of times, we noticed that it had developed a pronounce bend along its short edge. It seems incredibly easy to bend, and you may even physically break your $1,500 device without realizing that you are doing so. When we bent it back into the original shape, we discovered that it bends like warm butter. No other device in our lab has exhibited the same tendency.

iPhone performance

The performance of the smaller iPhone 12 Pro is admirable. It was able to match the rendering times and tracking of the iPad Pro models. However, the iPhone seems to have a very low threshold for allocation of maximum memory, and it gives up quickly when processing large models. Although the 2020 and 2021 iPad Pro render models with ease, and Android devices take their time but eventually do the job, the iPhone calls it quit by shutting down the app.

Samsung Galaxy Note 10+

We were surprised by what a contradictory device the Samsung Galaxy Note 10+ turned out to be. On the one hand, it is one of the few devices that support LiDAR-equivalent ToF for AR. The phone itself is fast and delightful. The surface detection in AR is excellent, and the device is almost on a par with iOS with respect to spatial tracking. However, the overall AR experience is dreadful. Even when the Note 10+ is coping with the lightest dataset, its frame rate is way below 60 FPS, and the experience feels clunky and unpolished.

LG V60 ThinQ 5G

Either Google or LG have misrepresented the capabilities of the LG V60 ThinQ. According to Google, it “supports time-of-flight (ToF) hardware depth sensor.” Our tests show that it does not. The LG V60 ThinQ 5G consistently performed worse at detecting surfaces than any other device. During the surface detection tests, it did a worse job than non-ToF Google Pixel phones, indicating not only that the V60 ThinQ 5G fails to engage the ToF sensor, but also that it is not as good as Pixel phones at processing video feed.