Skip to main content

XR8.addCameraPipelineModule()

XR8.addCameraPipelineModule(module)

Description

8th Wall camera applications are built using a camera pipeline module framework. For a full description on camera pipeline modules, see CameraPipelineModule.

Applications install modules which then control the behavior of the application at runtime. A module object must have a .name string which is unique within the application, and then should provide one or more of the camera lifecycle methods which will be executed at the appropriate point in the run loop.

During the main runtime of an application, each camera frame goes through the following cycle:

onBeforeRun -> onCameraStatusChange (requesting -> hasStream -> hasVideo | failed) -> onStart -> onAttach -> onProcessGpu -> onProcessCpu -> onUpdate -> onRender

Camera modules should implement one or more of the following camera lifecycle methods:

FunctionDescription
onAppResourcesLoadedCalled when we have received the resources attached to an app from the server.
onAttachCalled before the first time a module receives frame updates. It is called on modules that were added either before or after the pipeline is running.
onBeforeRunCalled immediately after XR8.run(). If any promises are returned, XR will wait on all promises before continuing.
onCameraStatusChangeCalled when a change occurs during the camera permissions request.
onCanvasSizeChangeCalled when the canvas changes size.
onDetachis called after the last time a module receives frame updates. This is either after the engine is stopped or the module is manually removed from the pipeline, whichever comes first.
onDeviceOrientationChangeCalled when the device changes landscape/portrait orientation.
onExceptionCalled when an error occurs in XR. Called with the error object.
onPausedCalled when XR8.pause() is called.
onProcessCpuCalled to read results of GPU processing and return usable data.
onProcessGpuCalled to start GPU processing.
onRemoveis called when a module is removed from the pipeline.
onRenderCalled after onUpdate. This is the time for the rendering engine to issue any WebGL drawing commands. If an application is providing its own run loop and is relying on XR8.runPreRender() and XR8.runPostRender(), this method is not called and all rendering must be coordinated by the external run loop.
onResumeCalled when XR8.resume() is called.
onStartCalled when XR starts. First callback after XR8.run() is called.
onUpdateCalled to update the scene before render. Data returned by modules in onProcessGpu and onProcessCpu will be present as processGpu.modulename and processCpu.modulename where the name is given by module.name = "modulename".
onVideoSizeChangeCalled when the canvas changes size.
requiredPermissionsModules can indicate what browser capabilities they require that may need permissions requests. These can be used by the framework to request appropriate permissions if absent, or to create components that request the appropriate permissions before running XR.

Note: Camera modules that implement onProcessGpu or onProcessCpu can provide data to subsequent stages of the pipeline. This is done by the module's name.

Parameters

ParameterTypeDescription
moduleObjectThe module object.

Returns

None

Example 1 - A camera pipeline module for managing camera permissions:

XR8.addCameraPipelineModule({
name: 'camerastartupmodule',
onCameraStatusChange: ({status}) {
if (status == 'requesting') {
myApplication.showCameraPermissionsPrompt()
} else if (status == 'hasStream') {
myApplication.dismissCameraPermissionsPrompt()
} else if (status == 'hasVideo') {
myApplication.startMainApplictation()
} else if (status == 'failed') {
myApplication.promptUserToChangeBrowserSettings()
}
},
})

Example 2 - a QR code scanning application could be built like this

// Install a module which gets the camera feed as a UInt8Array.
XR8.addCameraPipelineModule(
XR8.CameraPixelArray.pipelineModule({luminance: true, width: 240, height: 320}))

// Install a module that draws the camera feed to the canvas.
XR8.addCameraPipelineModule(XR8.GlTextureRenderer.pipelineModule())

// Create our custom application logic for scanning and displaying QR codes.
XR8.addCameraPipelineModule({
name: 'qrscan',
onProcessCpu: ({processGpuResult}) => {
// CameraPixelArray.pipelineModule() returned these in onProcessGpu.
const { pixels, rows, cols, rowBytes } = processGpuResult.camerapixelarray
const { wasFound, url, corners } = findQrCode(pixels, rows, cols, rowBytes)
return { wasFound, url, corners }
},
onUpdate: ({processCpuResult}) => {
// These were returned by this module ('qrscan') in onProcessCpu
const {wasFound, url, corners } = processCpuResult.qrscan
if (wasFound) {
showUrlAndCorners(url, corners)
}
},
})