ARKit

@denkeni
2023-06-14

Is AR only about placing objects?

Is AR only about face effects?

ARKit

iPhone X (2017)

ARKit 7

Apple Vision Pro (2023)

  • ARKit 1 (2017)／iOS 11
  • ARKit 2 (2018)／iOS 12
  • ARKit 3 (2019)／iOS 13
  • ARKit 4 (2020)／iOS 14
  • ARKit 5 (2021)／iOS 15
  • ARKit 6 (2022)／iOS 16
  • Spatial Computing (2023)／visionOS

AR is about understanding environment

AR is about sensors

iPhone 2G: accelerometer, proximity, ambient light

iPhone 3G: GPS

iPhone 3GS: compass

iPhone 4: gyroscope

.

.

.

iPhone 6: barometer

.

iPhone 7 Plus: telephoto camera

iPhone X: TrueDepth camera

.

iPhone 11: ultra wide camera

iPhone 12 Pro: LiDAR

.

.

Apple Vision Pro: IR camera and illuminator

Only Pro devices come with LiDAR

Since 2020

  • iPhone 12 Pro
  • iPad Pro 11-inch (2nd generation)
  • iPad Pro 12.9-inch (4th generation)

2023

  • Apple Vision Pro

Even Apple devices are fragmented

(Until Apple Vision Pro)

ARKit

Overview

  • ARKit 1 (20171)／A9 or later
  • ARKit 2 (20182)／A12
  • ARKit 3 (2019)
  • ARKit 4 (2020)／LiDAR
  • ARKit 5 (2021)
  • ARKit 6 (2022)
  • Spatial Computing (2023)／Vision Pro

  1. iPhone X (A11) was announced. ↩︎

  2. iPhone XS (A12) was announced. ↩︎

Front Camera

  • ARFaceTrackingConfiguration
  • TrueDepth Camera

Rear Camera

  • ARWorldTrackingConfiguration
  • Better with LiDAR
(ARKit is not available in iOS Simulator.)

Persistent AR Experience (1/2)

Save a world map to a file, use the same relocalization process to recover... even if your app has relaunched1.

  1. Managing Session Life Cycle and Tracking Quality ↩︎

Persistent AR Experience (2/2)

It's easy to successfully relocalize to a map recorded indoors under consistent artificial lighting1.

  1. Managing Session Life Cycle and Tracking Quality ↩︎

Shared Experiences (1/2)

Shared Experiences (2/2)

Detection (1/2)

Detection (2/2)

Face Tracking

Tracking and Visualizing Faces

Motion Capture

Capturing Body Motion in 3D (Bringing People into AR)

Location Anchor

Tracking Geographic Locations in AR

RoomPlan

Create a 3D model of an interior room by guiding the user through an AR experience

Miscellaneous

ARView / RealityView1
Object Capture1
Reality Composer (Pro)
AR Coaching View
Raycasting
sceneDepth
Text Annotations
Audio
App Clip Codes

  1. Available in RealityKit ↩︎

visionOS (xrOS)

Spatial Computing

Spatial Computing

  1. Elevate your windowed app for spatial computing
  2. Evolve your ARKit app for spatial experiences
  3. Design for spatial input

Apple Vision Pro

Work with Apple

  1. Compatibility evaluations
  2. Developer labs: Cupertino, London, München, Singapore, 上海, 東京.
  3. Developer kit

The future is in your hands.