handleMemoryPressure crashes apps on IOS with EXC_BAD_ACCESS

[dev-john-nguyen]

I believe this issue might be related to this discussion facebook/hermes#1518.

I'm working on an application that uses the react-native-vision-camera library (https://github.com/mrousavy/react-native-vision-camera) along with the useSkiaFrameProcessor hook to render drawings on camera frames. I'm encountering random crashes exclusively on iOS. The issue is difficult to reproduce in a another project, but it consistently occurs in this one, possibly due to the larger project size. The app works fine with useFrameProcessor, but we need the ability to draw on the frames. I tried using Skia Pictures to draw on the canvas to check if the issue was related to useSkiaFrameProcessor hook itself, but I’m still experiencing crashes. Any advice would be greatly appreciated!

Error log below.

Here's some code:

import {PaintStyle, Skia} from '@shopify/react-native-skia';
import React, {useEffect} from 'react';
import {
  Platform,
  StyleSheet,
  Text,
  NativeModules,
  NativeEventEmitter,
} from 'react-native';
import {
  Camera,
  useCameraDevice,
  useCameraPermission,
  useSkiaFrameProcessor,
  Frame,
  VisionCameraProxy,
} from 'react-native-vision-camera';
import {useSharedValue} from 'react-native-worklets-core';
const {HandLandmarks} = NativeModules;
const handLandmarksEmitter = new NativeEventEmitter(HandLandmarks);
const lines = [
  [0, 1],
  [1, 2],
  [2, 3],
  [3, 4],
  [0, 5],
  [5, 6],
  [6, 7],
  [7, 8],
  [5, 9],
  [9, 10],
  [10, 11],
  [11, 12],
  [9, 13],
  [13, 14],
  [14, 15],
  [15, 16],
  [13, 17],
  [17, 18],
  [18, 19],
  [19, 20],
  [0, 17],
];

const handLandMarkPlugin = VisionCameraProxy.initFrameProcessorPlugin(
  'handLandmarks',
  {},
);

export function handLandmarks(frame: Frame) {
  'worklet';
  if (handLandMarkPlugin == null) {
    throw new Error('Failed to load Frame Processor Plugin!');
  }
  return handLandMarkPlugin.call(frame);
}

function LiveFeed(): React.JSX.Element {
  const device = useCameraDevice('front');
  const {hasPermission, requestPermission} = useCameraPermission();
  const landmarks = useSharedValue({}) as any;

  useEffect(() => {
    // Set up the event listener
    const subscription = handLandmarksEmitter.addListener(
      'onHandLandmarksDetected',
      event => {
        if (event.landmarks.length) {
          landmarks.value = event.landmarks;
        } else {
          landmarks.value = {};
        }
      },
    );

    // Clean up the event listener when the component unmounts
    return () => {
      subscription.remove();
      console.log('removed');
    };
  }, []);

  useEffect(() => {
    requestPermission();
  }, [requestPermission]);

  const paint = Skia.Paint();
  paint.setStyle(PaintStyle.Fill);
  paint.setStrokeWidth(1);
  paint.setColor(Skia.Color('red'));

  const linePaint = Skia.Paint();
  linePaint.setStyle(PaintStyle.Fill);
  linePaint.setStrokeWidth(3);
  linePaint.setColor(Skia.Color('red'));

  const frameProcessor = useSkiaFrameProcessor(frame => {
    'worklet';
    frame.render();
    handLandmarks(frame);
    if (landmarks.value[0]) {
      const hand = landmarks.value[0];
      const frameWidth = frame.width;
      const frameHeight = frame.height;

      for (const [from, to] of lines) {
        frame.drawLine(
          hand[from].x * Number(frameWidth),
          hand[from].y * Number(frameHeight),
          hand[to].x * Number(frameWidth),
          hand[to].y * Number(frameHeight),
          linePaint,
        );
      }

      // Draw circles
      for (const mark of hand) {
        frame.drawCircle(
          mark.x * Number(frameWidth),
          mark.y * Number(frameHeight),
          6,
          paint,
        );
      }
    }
  }, []);

  if (!hasPermission) {
    return <Text>No permission</Text>;
  }
  if (device == null) {
    return <Text>No device</Text>;
  }

  const pixelFormat = Platform.OS === 'ios' ? 'rgb' : 'yuv';
  return (
    <>
      <Camera
        style={StyleSheet.absoluteFill}
        device={device}
        isActive={true}
        frameProcessor={frameProcessor}
        pixelFormat={pixelFormat}
      />
    </>
  );
}

export default LiveFeed;

Project Dependencies

[email protected]
[email protected]
@shopify/[email protected]
[email protected]

[dev-john-nguyen]

Also updated the project to react-native 0.75.4 to see if that would fix the issue, but still seeing the crashing behavior.

Can see some leaking memory.

Added a log statement and this happened. It logged in the console and then it crashes with "com.facebook.react.JavaScript (10): signal SIGABRT"

[dev-john-nguyen]

Noticed every time I mount the component there's a spike in memory, but when I unmount it doesn't release. I tried to trigger gc on unmount, but no luck in releasing the memory.