Skip to main content

Lightship VPS

Lightship VPS Overview

With Niantic’s Visual Positioning System (VPS), developers can create rich Location AR experiences that overlay virtual content on the physical world. WebAR content can be anchored to real-world locations, enabling virtual objects to interact with their physical surroundings. This makes the AR experience feel more personal, meaningful, and immersive, and gives users new reasons to explore the world around them.

Note: VPS localization is not currently previewable in the Simulator.

Managing Locations

The Geospatial Browser can be accessed from within your Project by selecting the map icon in the left hand menu (annotated as #1 in the image below). On this page you will find a map view (#2) which you can use to search to find VPS-activated Locations. Selecting a VPS-activated location will display the 3D mesh of the location (#3) so you can verify you have selected the correct location and add it to your project (#4).

ConsoleGSB

When you add a VPS-activated Location to your project you will see the Location in the "Project Locations" table in the Geospatial Browser (annotated as #1 in the image below). Once you have a Location in the “Project Locations" table you can use the "Download" button (#2) to download a GLB or OBJ (toggle shown as #3) version of the 3D mesh and open it in third-party 3D software applications, such as Blender, or import it directly into your 8th Wall project. When referencing Locations in your project code you will need to copy the "Name" field (#4) from the "Project Locations" table.

ConsoleGSBManageWayspots

If the location you'd like to use in your project is not available as a VPS Location, you can create the location by following the instructions in the Create New Location section.

Create New Location

  1. Select a location on the map where you want to create a new VPS Location. (see VPS Location Requirements to learn more about choosing a good location to create a VPS Location).

  2. Create Location: Click the "Create Public Location" button to start the process to create a new VPS Location.

ConsoleCreateWayspot

  1. Check for Duplicates: Before creating a new Location, you are required to check that your Location doesn't already exist. Compare your desired Location to others already on the map to ensure that you are not creating a duplicate. If this is not a duplicate Location, you must check the "My Location is not a duplicate" box and click on the "Next" button to continue.

ConsoleCreateWayspotNoDuplicate

  1. Add Location Information: Location metadata will be visible to developers using the Geospatial Browser and can be visible to end-users. Remember that Niantic's Trust & Safety team uses the information you provide to determine whether the Location meets our criteria to be made publicly available. Once you have added the following information for the Location you are trying to create, click on the "Submit" button:
  • Title (125 characters)
  • Description (250 characters)
  • Category (1 or more)
  • Image (if available)
  1. Your location should immediately be added to your Location Submissions tab in the Geospatial Browser with the type "Public" and the status "Not Activated". It will be available for scanning within a few minutes and can be activated once it’s fully scanned.

Installing Niantic Wayfarer

In order to scan new VPS Locations or add scans to previously activated Locations, you'll need to install the Niantic Wayfarer app on your phone. Please see below for installation and usage instructions.

iOS

The Niantic Wayfarer App requires iOS 12 or later and an iPhone 8 or later. A LiDAR-capable device is not required.

To install the Niantic Wayfarer App, go to Testflight for Niantic Wayfarer (8th.io/wayfarer-ios) on your iOS device.

Android (Beta)

The Niantic Wayfarer App requires the ARCore package.

To install the Niantic Wayfarer App, go to Niantic Wayfarer (8th.io/wayfarer-android) on your Android device.

Using Niantic Wayfarer

You can add scans to Public Locations as well as create Test Scans with the Niantic Wayfarer App.

Once you have installed the app, login with your 8th Wall credentials by pressing the Login with 8th Wall button.

If you have access to multiple workspaces, select a workspace by pressing the 8th Wall Workspace dropdown on the profile page.

Login PageProfile Page
wayfarer app loginwayfarer app profile

On the Map page, select a VPS Location to add a scan to a public Location (1), or select Scan to add a test scan to your workspace (2).

Take a scan of the area using the recommended scanning technique.

Map PageScanning Page
wayfarer add scanwayfarer scanning

Once the scan has been completed, select either public or test, and then upload.

Scan TypeScan Upload
wayfarer scan typewayfarer scan upload

Processing scans can take 15-30 minutes. Once processed, scans will populate in the geospatial browser.

Issues related to scanning or processing should be directed to support@lightship.dev.

You can find more information on how to use the Wayfarer app in the Lightship documentation.

Scanning Technique

Scanned VPS-activated locations should be no larger than a 10-meter diameter around the location. For example, a typical statue would work as a VPS-activated Location. An entire building, however, would not. One face or doorway/entrance into a building might work. We recommend sticking with smaller areas for starters (e.g. a desk, statue, or mural).

Before scanning, be aware of your surroundings and ensure you have the right to access the location you are scanning.

  1. Check the area to be scanned and the surroundings of the scanned object to determine if there are any obstacles and to select a scanning route. It is necessary to plan the route you intend to use for scanning before starting the procedure.

  2. Make sure your camera is in focus. Camera shake can negatively affect 3D reconstruction. Keep your phone as close to your side as possible to avoid blurring. Walk around the object you are scanning instead of standing in one location and moving your phone.

  3. Walk at a slow and natural stroll pace. Move slowly and smoothly during scanning. Sudden changes of direction are a definite no-no. Move slowly and smoothly with your feet on the ground. If you are scanning in a darker setting, it’s even more important to move slowly and smoothly. Move the phone with you while you are moving (think crab walk).

  4. VPS Location should always be the focal point. In order for us to build the map, it's important to focus on the VPS Location and capture the full 360° orbit of it. If it is not safe or not possible to get 360° coverage, capture as much as you can.

  5. Vary your distance/angles (0-10m or 0-35ft). In order for the 3D map to work well in different scenarios, it’s important that we capture the environment around the Location and have a variety of different scans. It’s important to vary your distance and angles while scanning the Location.

Video of recommended VPS Location scanning technique:

Things to avoid while scanning

  1. Avoid scanning while the surroundings are not safe, e.g. in the middle of the road, or in a playground with children.

  2. Avoid scanning while the Location is too far away (>10m or 35ft) or too big to focus your camera on.

  3. Avoid scanning while you are casually taking a walk or jogging. It is important to keep the Location as your focal point at all times.

  4. Avoid pointing your phone at very bright objects such as a fluorescent light or the sun.

  5. Avoid not moving or moving too fast while scanning. Abrupt motions will cause offsets in the reconstruction.

  6. Avoid scanning if your phone gets too hot. If the temperature of the device rises too high, the performance of the device will be greatly reduced, which will negatively affect the scan.

  7. Avoid uploading any scans that look incomplete or not representative of what you're trying to scan.

Test Scans

Test scans are a single mesh, available to only one workspace, to develop and test VPS experiences. While test scans are a great solution for developing and testing VPS experiences while a public Location is being nominated or activated, they are not authorized for use in published projects.

Test scans are created using the Niantic Wayfarer app. Ensure you’re logged in to Wayfarer using 8th Wall credentials and that the correct workspace is selected from the Profile page. The test scan will only be available in the selected 8th Wall workspace at the time of scanning and uploading. Scans can not be moved to a different workspace or Lightship account.

In the Wayfarer app, select Scan and take a scan of the area.

Test scans should be 60 seconds or less; a new mesh is generated every 60 seconds – so scanning for 120 seconds will result in 2 test scans. All test scans are unaligned.

Once processed, you can preview the mesh and add it to your project from the geospatial browser Test Scans tab.

Test scans tab

If your test scan fails processing, you may need to rescan. Reach out to support@lightship.dev for more information.

Location Types

In the Geospatial Browser, you will see four different types of Locations:

TypeIconDescription
PublicWSPublic"Public" Locations have been approved by Niantic's Trust & Safety team and have met the required criteria of safety and public accessibility. These Locations may be used in published projects.
PendingWSPending"Pending" Locations are being reviewed by Niantic's Trust & Safety team to determine if they meet the required criteria of safety and public accessibility. This process can take up to 2 business days. Pending Locations can be scanned and activated while waiting for the review to complete.
RejectedWSRejected"Rejected" Locations may have failed Niantic's Trust & Safety review, be a duplicate of an existing or previously rejected Locations, or may not be allowed by Niantic for another reason. These Locations cannot be added to projects.
TestWSTest"Test" Locations are only accessible to your Workspace by scanning the location using Niantic's Wayfarer app. Test Locations are intended for use during development and may not be included in a published project.

For questions or issues related to creating VPS Locations, or to check the status of existing and existing Location, please contact support@lightship.dev

Location Status

In the Geospatial Browser, you will see five different statuses for VPS Locations:

StatusIconDescription
Not ActivatedWSNotActivatedLocations with a status of 'Not Activated' have not had any scans submitted for the location. A minimum of 10 viable scans must be submitted for the location before you will be able to request activation. After one scan is submitted the Location status will change to 'Scanning'.
ScanningWSScanningLocations with a status of 'Scanning' have had at least one scan submitted for the location. A minimum of 10 viable scans must be submitted for the location before you will be able to request activation.
ProcessingWSProcessingLocations with a status of 'Processing' have had an activation request submitted and will display the 'Processing' status until the activation process has completed. Typically, an activation request is completed within 4 hours. You will receive and email when the process is complete.
ActiveWSActiveLocations with a status of 'Active' are available to be used in projects to create WebAR content using VPS for Web.
FailedWSFailedLocations with a status of 'Failed' encountered an issue during the activation process. This could be a result of a number of factors, such as poor suitability of the location for VPS, insufficient scans, or corrupt data. Unfortunately this means that this Location cannot be used to create WebAR content using VPS. We encourage you to find a new Location to use in your 8th Wall project.

For questions or issues related to Location scanning, activating or status, please contact support@lightship.dev

Location Requirements

Locations will only be approved and made publicly available if they meet the following criteria:

  • Are a permanent physical, tangible, and identifiable place or object, or object that placemarks an area.
  • Are safe and publicly accessible by pedestrians (indoor or outdoor).
  • Contain accurate information in the title, description, and photo.

Locations perform better on VPS, when they also meet the following criteria:

  • Have a clear focal point that fits within a 10m diameter sphere.
  • Have features that are distinct and consistent in appearance. (Ex. A sandy beach, or a crowded patio space with moveable furniture will not work well.)
  • Have minimal features that are reflective or transparent.

Location Quantities

There is no limit to the number of Locations that can be associated with an 8th Wall project. Locations are localized server side via the VPS service.

Location Events

8th Wall emits events at various stages in the Project Location lifecycle (e.g. scanning, found, updated, lost, etc). Please see the API reference for specific instructions on handling these events in your web application:

Location Quality

After a Location has been VPS activated, Niantic provides a quality rating in the Geospatial browser. Location details display either Fair Quality or Good Quality.

Location Quality refers to the Location's ability to localize at any time. Locations with several scans in all types of lighting tend to have a higher quality. Locations with minimum required scans or a majority of scans in one type of lighting tend to have a lower quality.

Quality rating is an automated process and may not reflect the actual performance of the Location. The best way to determine quality is to try it out yourself.

Location Alignment

The unaligned warning can happen for various reasons and means localization against the mesh can not be guaranteed. Although the mesh may work well for localization, the warning indicates the mesh is experimental and should be used at your own risk.

Note: All test scans are unaligned.

Enabling Lightship VPS

To enable VPS in your WebAR project, you'll need to set enableVPS to true.

For A-Frame projects, set enableVps: true on the xrweb component on the <a-scene>

For Non-AFrame projects, set enableVps: true in the call to XR8.XrController.configure() prior to engine start.

Example - AFrame

<a-scene
coaching-overlay
landing-page
xrextras-loading
xrextras-runtime-error
...
xrweb="enableVps: true;">

Example - Non-AFrame

XR8.XrController.configure({enableVps: true})
// Then, start the 8th Wall engine

Developing Bespoke VPS Experiences

Bespoke VPS scenes are designed for a single Location and utilize a reference mesh from the Geospatial Browser to align AR content.

Part 1: Add Location to scene

  1. Open the Geospatial Browser (map icon 🗺 on the left)
  2. Find a VPS-activated Location (or nominate/activate your own)
  3. Add the Location to the project

Part 2: Use Location GLB as reference for custom AR animation

  1. Download the reference GLB from the right side of the row.
  2. Use this in your 3D modeling software (Blender, Maya, A-Frame, etc) to position AR content relative to the mesh origin.

IMPORTANT: The origin of this 3D model is the origin of the Location. DO NOT RESET THE ORIGIN OR YOUR CONTENT WILL NOT BE ALIGNED.

OPTIONALLY: If the mesh downloaded from the Geospatial Browser is not high enough quality to use for a baked animation, physics, or occluder material, you might consider taking a scan using a third-party application like Scaniverse and aligning that high-quality mesh with the one downloaded from the Geospatial Browser.

  1. Import animation GLB into Cloud Editor and add to scene
  2. Add the named-location component to your asset's <a-entity>. The 'name' attribute refers to the Project Location's name from the Geospatial Browser.

Ta-da! 🪄 Your animation should appear aligned to the Location in the real world.

Part 3: Adding occlusion and shadows

  1. In your scene, add <a-entity named-location="name: LOCATIONNAME"><a-entity>
  2. Add three <a-entity> inside this element as its children. These will be your occluder mesh, shadow mesh and VPS animation.
  3. In the first <a-entity>, add xrextras-hider-material and gltf-model="#vps-mesh". "#vps-mesh" should refer to a version of your reference GLB that has had its textures removed and geometry decimated.
  4. In the second <a-entity>, add shadow-shader, gltf-model="#vps-mesh", and shadow="cast: false". The shadow shader applies a shadow material to the reference mesh with a polygon offset to prevent Z-fighting. You can choose whether you want the vps-mesh to cast a shadow on the real world with shadow="cast: true".
  5. In the third <a-entity>, add gltf-model="#vps-anim", reflections="type: realtime", play-vps-animation and shadow="receive:false". play-vps-animation waits until the vps-coaching-overlay has disappeared before playing the VPS animation.

Remote Desktop Development Setup

It is often helpful to use the A-Frame inspector to position content remotely on your desktop. To set up this project's scene for remote desktop development, disable the following components by adding a letter to the beginning (i.e. "Znamed-location"):

  • xrweb -> Zxrweb
  • xrextras-loading -> Zxrextras-loading
  • named-location -> Znamed-location
  • xrextras-hider-material -> Zxrextras-hider-material

Now you can open the A-Frame Inspector (Mac: ctrl + opt + i, PC: ctrl + alt + i) and position content relative to the VPS mesh imported from the Geospatial Browser. Remember: this is an inspector. You will need to copy the transform values back into your code.

Optionally, you can temporarily reposition the <a-entity named-location> to the center of the scene to assist with iteration speed. NOTE: reset <a-entity named-location> to position="0 0 0" to ensure VPS content is aligned correctly.

Remote Mobile Development Setup

It is often helpful to use the A-Frame inspector to simulate VPS remotely on your mobile device. To set up this project's scene for remote mobile development, disable the following components by adding a letter to the beginning (i.e. "Znamed-location"):

  • named-location -> Znamed-location
  • xrextras-hider-material -> Zxrextras-hider-material

Next, you'll need to disable VPS and enable absolute scale. This will ensure the reference mesh is sized correctly for accurate simulation:

xrweb="enableVps: false; scale: absolute;"

You should temporarily reposition the <a-entity named-location> to the center of the scene to assist with iteration speed. Try to align the base of your reference mesh with y="0" (the ground). NOTE: Before deploying your VPS project, reset <a-entity named-location> to position="0 0 0" to ensure VPS content is aligned correctly.

Developing Procedural VPS Experiences

Procedural VPS scenes are designed to use any detected Location (as opposed to specific Project Locations). Once detected, the Location's mesh is available to you to generate procedurally-generated VPS experiences.

There are two procedural-related events emitted by the 8th Wall engine:

  • xrmeshfound: emitted when a mesh is first found either after start or after a recenter()
  • xrmeshlost: emitted when recenter() is called.

After a mesh is detected, the 8th Wall engine will continue to track against that mesh until recenter() is called.

VPS FAQ

What is Lightship VPS?

Lightship VPS (Visual Positioning System) is a cloud service that enables applications to localize a user's device at real-world locations, enabling users to interact with persistent AR content and powering new immersive experiences. VPS determines the device's position and orientation (pose) by referencing map data that exists in Niantic's cloud.

How does VPS work?

When a device makes a call to the VPS service, the service receives a query image from the user's device along with their rough location (from GPS) as inputs and attempts to localize them using the map(s) that exist at that location. If localization is successful, then the service returns the device's position and orientation (pose) corresponding with the timestamp of the image that was transmitted. Because there is a time delay between when a VPS query image is captured and when a response is received from the VPS service, the device needs to have a motion tracking system in order to stay accurately localized while moving. When the VPS service returns a pose estimate to the device, the difference in pose from the device's tracking system is added to the localization response so that VPS can “keep up” with how the device moved while waiting for the server’s response to the VPS query.

What is a scan?

AR scans from players, developers, and surveyors are the fundamental ingredient used to create the Niantic Map: Niantic’s 3D map of the world. AR scans are recorded and uploaded using Niantic's AR scanning framework, which is a module used inside Pokemon Go, Ingress, and the Wayfarer App. Each AR scan consists of a series of video frames with supporting data from accelerometers and GPS sensors that construct a 3D model of the world from multiple 2D images. AR scans are used by Niantic to build maps and meshes of real-world locations.

What is a map?

In VPS parlance, a map is the data artifact that is used to localize your device when the VPS API is called. A map can be thought of as a function that takes a query image as input and then returns position and orientation (pose) as output. The map that corresponds to a given location is created from the scans that were uploaded at that location. VPS maps are not human-readable.

What is a mesh?

In VPS parlance, a mesh is a 3D model of a real-world location or object. Meshes provide a detailed representation of a physical space or object, and are useful for understanding what a location looks like, as a reference for authoring AR content, and for creating physics and occlusion effects. Like maps, meshes that correspond to a given location are created from the scans that were uploaded at that location. Meshes are both human- and machine-readable.

Where can I use VPS?

VPS is available at over 150,000 real-world locations, with more locations being added every day. In order for a location to be available on VPS, a sufficient amount of AR scan data must be uploaded at that location and the VPS activation process must be completed. Developers can add new locations to the map and request VPS activation of fully scanned locations using the Geospatial Browser.

How does VPS activation work?

For a location to be eligible for VPS activation, it must have at least 10 scans uploaded that pass minimum quality checks, and the time difference between the oldest and newest scans at the location must be at least 5 hours. These requirements ensure that the resulting maps and meshes are of sufficient quality and capture enough variation such that users will be able to localize reliably. The VPS activation process runs on Niantic's AR mapping infrastructure and involves many complex steps. From the pool of eligible scans at the location, an algorithm selects most of the scans to use for building maps and meshes, and the remaining handful for validation and measuring localization quality. The activation process for a location runs on Niantic servers and usually takes 1-2 hours to complete.

Can I find my scans after VPS activation is done?

During the activation process, the maps and meshes created from the uploaded scans are fused together in order to incorporate as much information as possible. The final product, which is used by developers to author content and by users to localize, consists of scans from many different sources. Scan data are mixed together to create a more comprehensive representation of the location, so there is not a one-to-one relationship between the scans that are uploaded at a location and the maps and meshes that area created once it is VPS-activated.

Can I add more scans to a location that's already activated?

In some cases, developers may wish to add additional scans to a location that was previously activated in order to improve the quality and the coverage of the location's maps and meshes. In order for a Location to be eligible for "reactivation," it must have had at least 5 additional scans uploaded since the last time it was activated. Importantly, it is not yet possible to add new scans to an existing fused map, rather, the process of reactivating requires a new fused map to be built that incorporates the new scans in the context of the existing ones.

How do I request VPS activation of a new location?

Once a location has enough scans uploaded to meet the VPS activation requirements (at least 10 total scans with at least a 5-hour time difference between the oldest and newest scans), developers can request VPS activation by selecting the location in the Wayfarer App or the Geospatial Browser and pressing the “activate” button. This will add the location to the activation queue. Typically, an activation request is completed within 2 hours. Developers also have the option to request reactivation of an existing location once 5 additional scans are uploaded.

Does VPS work at night or in poor weather conditions?

VPS works best when there is good visibility. In order to maximize the likelihood of successful VPS-powered experiences, it is best to upload many AR scans that cover a wide swath of different conditions (e.g. different times of day, different weather conditions, etc.). For example, if you are building an experience in a location that gets a lot of rain, having some scans from a rainy day is very helpful.

Do AR scanning and VPS require phones with LiDAR sensors?

AR scanning and VPS do not require LiDAR.