Interface: Frame
A single frame, as seen by the camera. This is backed by a C++ HostObject wrapping the native GPU buffer. At a 4k resolution, a raw Frame can be 12MB in size.
Example
const frameProcessor = useFrameProcessor((frame) => {
'worklet'
console.log(`Frame: ${frame.width}x${frame.height} (${frame.pixelFormat})`)
}, [])
Hierarchy​
-
Frame
↳
DrawableFrame
Properties​
bytesPerRow​
• Readonly
bytesPerRow: number
Returns the amount of bytes per row.
Defined in​
types/Frame.ts:33
height​
• Readonly
height: number
Returns the height of the frame, in pixels.
Defined in​
types/Frame.ts:29
isMirrored​
• Readonly
isMirrored: boolean
Returns whether the Frame is mirrored (selfie camera) or not.
Defined in​
types/Frame.ts:41
isValid​
• Readonly
isValid: boolean
Whether the underlying buffer is still valid or not.
A Frame is valid as long as your Frame Processor (or a runAsync(..)
operation) is still running
Defined in​
types/Frame.ts:21
orientation​
• Readonly
orientation: Orientation
Represents the orientation of the Frame, relative of what the desired output orientation is.
For example, if the phone is held in 'portrait'
mode and the Frame's orientation
is 'landscape-left'
, it is 90° rotated relative to the phone's rotation.
To make the frame appear up-right, one would need to counter-rotate it by 90°. Such counter-rotations should not actually rotate pixels in the buffers, but instead be handled via flags or transforms to avoid any performance overheads.
For example in MLKit, the caller just needs to pass the Frame's orientation
to it's detect(...)
function and it will interpret buffers in that target orientation.
See
See "Orientation"
Defined in​
types/Frame.ts:61
pixelFormat​
• Readonly
pixelFormat: PixelFormat
Represents the pixel-format of the Frame.
Defined in​
types/Frame.ts:65
planesCount​
• Readonly
planesCount: number
Returns the number of planes this frame contains.
Defined in​
types/Frame.ts:37
timestamp​
• Readonly
timestamp: number
Returns the timestamp of the Frame relative to the host sytem's clock.
Defined in​
types/Frame.ts:45
width​
• Readonly
width: number
Returns the width of the frame, in pixels.
Defined in​
types/Frame.ts:25
Methods​
getNativeBuffer​
â–¸ getNativeBuffer(): NativeBuffer
Get the native platform buffer of the Frame.
- On Android, this is a
AHardwareBuffer*
- On iOS, this is a
CVPixelBufferRef
The native buffer needs to be manually deleted using
NativeBuffer.delete()
, and this Frame
needs to be kept alive as long as-, or longer than
the NativeBuffer
.
Returns​
Defined in​
types/Frame.ts:106
toArrayBuffer​
â–¸ toArrayBuffer(): ArrayBuffer
Get the underlying data of the Frame as a uint8 array buffer.
The format of the buffer depends on the Frame's pixelFormat
.
Note that Frames are allocated on the GPU, so calling toArrayBuffer()
will copy from the GPU to the CPU.
Returns​
ArrayBuffer
Example
const frameProcessor = useFrameProcessor((frame) => {
'worklet'
if (frame.pixelFormat === 'rgb') {
const buffer = frame.toArrayBuffer()
const data = new Uint8Array(buffer)
console.log(`Pixel at 0,0: RGB(${data[0]}, ${data[1]}, ${data[2]})`)
}
}, [])
Defined in​
types/Frame.ts:87
toString​
â–¸ toString(): string
Returns a string representation of the frame.
Returns​
string
Example
console.log(frame.toString()) // -> "3840 x 2160 Frame"
Defined in​
types/Frame.ts:95