Visual Positioning System SDK
and API for Any Camera

Centimeter-accurate 6-DoF pose for phones, glasses, headsets and robots,
from the scans you already have.
Add absolute pose to any tracking stack

ARKit, ARCore, Quest passthrough or your robot's odometry keeps tracking. MultiSet adds a 6-DoF pose with a confidence score in the map's frame, and relocalizes on its own after a tracking break.

SDKs for Unity, iOS, Android, WebXR and Quest

Install the SDK for your platform, or call the REST API from anything with a camera.
Unity SDK
Map, localize and track objects from one UPM package. Deep Search, NavMesh and multiplayer samples included.
Native iOS SDK
Swift for iOS 16+. Map with or without LiDAR, then localize and track objects in ARKit.
Native Android SDK
Kotlin for API 28+. Feed frames from any camera, ARCore or not, XREAL Aura included.
WebXR SDK
Localize in the browser with @multisetai/vps. Three.js and Needle adapters, Chrome on Android.
Quest SDK
Localize Quest 3 and 3S from the passthrough camera. Object tracking included.
REST API and Robots
Send any camera frame over HTTPS. There's a Python client and a Unitree Go2 adapter on GitHub, and ROS 2 is a supported SDK target.

Real2Sim2Real on one map
Build against a copy of the site, then run on the real one.

Simulate the site before you ship

Every new map exports a metric GLB and USDZ in the frame the API returns poses in. Plan routes in Unity or test in NVIDIA Isaac Sim, and the real robot lands on them. Splat scenes with collision meshes are what's next.

MultiSet VPS SDK
One map for every device. Deep Search for the hard views.

View through XREAL Aura glasses as a Unitree Go2 walks the route the wearer set
Any Device
Phones, glasses and robots on one map
A robot, a technician's glasses and a phone localize against the same scan and report positions you can compare directly. Adding a platform means adding a client against the map you already have.
Unity, iOS, Android, WebXR and Quest SDKs
Ray-Ban and XREAL Aura samples
Live shared poses for multiplayer
REST for any other camera
Read the guide
A LaMAR phone photo placed in its building by MultiSet Gen2.5, 0.9 cm and 0.08° from ground truth
Deep Search
Land the views a fast query misses
Deep Search searches more of the map for corridors, basements and repeating floors, with up to 15% higher recall. With it, Gen2.5 places 79.74% of LaMAR phone photos within 10 cm and 1° on the validation split, more than any published method.
Single frame, 4 to 6 frames, or 360
Engine chosen per request
Pose check against device tracking
First calls take up to 90% less time
Read the guide
Geo-fences of three mapped sites, with a coordinate query resolving to the one that covers it
By Coordinate
Localize by GPS coordinate
Send a latitude and longitude in place of a map code, and the server picks the georeferenced map that covers you. Poses can also come back in WGS 84, and OSCP clients get a GeoPose endpoint.
Nearby-map lookup by geo-fence
Hint radius from 5 to 100 m
OSCP GeoPose 1.0 and 2.0
Control points, EPSG or Helmert
Read the guide
Token request, localization query and JSON pose response from the MultiSet REST API
REST API
A frame in, a pose out
Swap a client ID and secret for a 30-minute token, post a frame with its intrinsics, and get back position, rotation and confidence. Signed webhooks fire when a map finishes processing.
Read, Query, Write and Delete scopes
Single, multi-frame and 360 queries
Resumable uploads up to 25 GB
12 signed webhook events
Read the API reference
Three maps built from an E57 laser scan, a 360 video and a phone without LiDAR
Capture with what you have
• E57 from Leica, NavVis, FARO, XGRIDS, Matterport
• 360 video from handheld cameras and drones
• Gaussian splats as PLY, SPZ or LCC2
• Phones with or without LiDAR
Four phone scans of one office joined into a single MapSet
Merge every scan into one frame
• MapSets by overlap, gap or georeference
• One frame across floors and buildings
• Control points, EPSG codes or Helmert
• Share links that expire on schedule
One view of a brewery map as textured mesh, raw mesh and point cloud, labelled with export formats
Take the site into simulation
• Metric GLB at the pose origin
• USDZ for NVIDIA Isaac Sim and Omniverse
• Splats and 5 cm point clouds
• Collision-ready splat scenes on the roadmap
A town captured with a 360 camera, then the same town in NVIDIA Isaac Sim, where a simulated drone localizes from its own camera frames
Test before anyone travels
• Replay site captures in the Unity Editor
• A NavMesh baked once that's ready on site
• Place content against real geometry
• Run simulated cameras against the real map
Phone app showing a localized Unitree Go2 on its route to a picked spot
Localize the real robot
• Phones, glasses, drones and robots
• A confidence score on every pose
• Map-to-odom correction for robots
• On-device on iOS and Android
The same office viewpoint on 28 May, 18 Jun, 20 Aug and 10 Sep, from bare concrete shell to finished fit-out, matched across six captures.
See what changed between scans
• Re-scan on any cadence, with any hardware
• Every scan lands in one frame, as a time series
• Compare any two dates, viewpoint by viewpoint
• Anchors keep their coordinates
Signed webhook request for a map that finished processing
Wire it into your pipeline
• Scoped machine-to-machine tokens
• Signed webhooks on 12 events
• Resumable uploads up to 25 GB
• Analytics API for your dashboards

Run where your data lives

Self-Hosted
On your servers, on-prem or air-gapped. Maps and imagery don't leave your network.
MultiSet Cloud
Managed, autoscaled and SOC 2 Type II. 99.95% uptime target on paid plans.
Private Cloud
Single-tenant on AWS, Azure or GCP, in your region of choice, with custom SLAs.
On-Device
Localize on iOS and Android with no connection. Other GPU, CPU and NPU targets on request.
Accuracy
Sub-5 cm median
Multi-frame
4 to 6 images
Hint radius
5 to 100 m
Token
30 min
Uploads
Up to 25 GB
Webhooks
12 events
iOS
16+
Android
API 28+
Smart glasses

Ray-Ban Display and XREAL Aura

Stream the Ray-Ban camera to the paired phone through Meta's Wearables Device Access Toolkit, localize, and guide the wearer by audio or on-lens HUD. On Android XR, the SDK 2.1.0 sample runs on XREAL Aura.

Which SDK for which platform

PlatformLanguageLocalizationObject trackingOn-device
UnityC#Single frame, multi-frame, Deep SearchYesYes, Enterprise
iOS nativeSwiftSingle frame, multi-frame, Deep SearchYesUnity package only
Android nativeKotlinSingle frame, multi-frame, Deep SearchYesUnity package only
Android XR (XREAL Aura)KotlinSingle frame, multi-frameYesNo
WebXRJavaScriptSingle frameYesNo
Meta Quest 3 and 3SC# via UnitySingle frame with Deep Search, multi-frameYesNo
Meta Ray-BanSwift and Kotlin samplesSingle frame, repeated fixesNoNo
Robots and any cameraPython or RESTSingle frame, multi-frame, 360YesServer-side

ROS 2 is a supported SDK target. If your platform isn't listed, the REST API takes a frame from any camera.

A visual localization API that returns a 6-DoF pose

Send one image or four to six frames to the visual localization API. Get back a 6-DoF pose and a confidence score.

Most teams localize the same day.

StepDo this
1. MapScan with the MultiSet app, or upload an E57, MatterPak, splat or 360 video
2. AuthenticateSwap your client ID and secret for a 30-minute token
3. LocalizePost a frame and its camera intrinsics
4. GateSkip low-confidence poses before you place content
Request
# 1. Token, valid 30 minutes
curl -X POST https://api.multiset.ai/v1/m2m/token \
  -H "Authorization: Basic $(printf '%s' "$CLIENT_ID:$CLIENT_SECRET" | base64)"

# 2. Localize one frame
curl -X POST https://api.multiset.ai/v1/vps/map/query-form \
  -H "Authorization: Bearer $TOKEN" \
  -F "mapCode=MAP_RJFKKWQ1787J" -F "isRightHanded=false" \
  -F "width=720" -F "height=960" \
  -F "fx=670.462" -F "fy=670.462" -F "px=478.839" -F "py=365.347" \
  -F "queryImage=@query.jpg"
Response
{
  "poseFound": true,
  "position": { "x": -2.877, "y": 1.402, "z": 7.677 },
  "rotation": { "x": -0.003, "y": 0.675, "z": -0.003, "w": 0.738 },
  "confidence": 0.918,
  "mapCodes": ["MAP_RJFKKWQ1787J"]
}

Your client ID and secret come with a Free account at developer.multiset.ai, and you won't need a card.

Frequently asked questions
Which SDKs and platforms does MultiSet support?

Unity, native iOS and Android, WebXR and Meta Quest, plus samples for Ray-Ban and XREAL Aura glasses. Any other camera uses the REST API. Robots get a Unitree Go2 adapter over WebRTC, and ROS 2 is a supported SDK target.

How do I localize my first device?

Create a Free account, with no card needed. Scan a space with the MultiSet app on any ARKit or ARCore phone, or upload a scan. Open your platform's sample, point it at the map and localize. The Free plan includes 1,000 API calls.

How accurate and how fast is localization?

Sub-5 cm median 6-DoF accuracy, an SLA under 10 cm and under 1 cm of drift at 10 m. A cloud query takes 1.5 to 2.5 s for one frame and 2.7 to 3.5 s for four to six. On-device runs in about 52 ms.

Which formats can I import and export?

Import E57 from five scanner brands, MatterPak, Gaussian splats, 360 video from handheld cameras and drones, and phone scans with or without LiDAR. Export GLB and USDZ meshes, PLY and SPZ splats and a 5 cm PCD point cloud.

Can I run MultiSet on-device or in my own infrastructure?

Yes, on Enterprise. On-device localization runs on iOS and Android and suits air-gapped sites. Enterprise also covers dedicated cloud, private cloud and on-prem. Self-serve plans run on MultiSet Cloud.

Does a VPS SDK replace ARKit, ARCore or SLAM?

ARKit, ARCore or the robot's odometry keeps tracking frame to frame. VPS adds the absolute fix that cancels drift and puts every device in the map's frame. Robots apply it as a map-to-odom correction.

Should I use an SDK or the REST API?

Use an SDK on Unity, iOS, Android, WebXR or Quest, since it handles capture, intrinsics and tracking breaks. Use the REST API for robots, drones, fixed cameras and backend jobs.

Can the same map drive a robot simulation and the real robot?

Yes. Maps processed since September export GLB and USDZ at the origin the API returns poses in, metric and Y-up. Plan in NVIDIA Isaac Sim or Unity, and the robot localizing on the same map lands on your routes.

Can I track a physical object without markers?

Yes. Upload a textured, metric GLB of a rigid object and every SDK can track it, with up to 10 objects in one query. Anchor instructions to a machine or a part.

Which companies offer a VPS SDK for GPS-denied AR and robotics?

Compare four things: mapping from scans you already have, accuracy indoors where GPS fails, SDK coverage for your devices and robots, and where it can run. MultiSet maps from any scan, holds sub-5 cm indoors and runs from cloud to device.