Skip to content

Commit

Permalink
Synchronous and external trigger documentation.
Browse files Browse the repository at this point in the history
Added files to documentation about how to externally trigger IMX296
and information about synchronising cameras for IMX477 & IMX296
  • Loading branch information
Ben Benson committed Aug 21, 2023
1 parent 4e0a787 commit 100920b
Show file tree
Hide file tree
Showing 6 changed files with 179 additions and 0 deletions.
67 changes: 67 additions & 0 deletions documentation/asciidoc/computers/camera/external_trigger.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
== External Trigger on the Global Shutter Camera
The global shutter camera can be triggered externally by pulsing the external trigger (denoted on the board as XTR) connection on the board.
Multiple cameras can be connected to the same pulse, allowing for an alternative way to synchronise two cameras.

The exposure time is equal to the low pulse-width time plus an additional 14.26us. i.e. a low pulse of 10000us leads to an exposure time of 10014.26us.
Framerate is directly controlled by how often you pulse the pin. A PWM frequency of 30Hz will lead to a framerate of 30 frames per second.

In this example, we will be using a Raspberry Pi Pico to create the pulses that will drive the camera modules.

image::images/external_trigger.jpg[Image showing pulse format]

=== Preparation

NOTE: When soldering to the Global Shutter Camera, please remove the plastic back cover to avoid damaging the cover.

**If you have the transistor Q2 fitted, shown in blue on image below, you will need to remove R11 on the board, shown in red.**
This connects GP0 to XTR and without removal, the camera will not operate
in external trigger mode.
The location of the resistor is displayed below.

image::images/resistor.jpg[Image showing resistor to be removed]

Next solder a wire to the touchpoint of XTR on the GS Camera board.
Connect these to the Pico - XTR to any pin. Pin 28 is used in this example. Ground is not needed to be connected.

==== Boot up the Raspberry Pi with the camera connected.

Enable external triggering through superuser mode:
[,bash]
----
sudo su
echo 1 > /sys/module/imx296/parameters/trigger_mode
exit
----

==== Raspberry Pi Pico Micropython Code
[,python]
```
from machine import Pin, PWM

from time import sleep

pwm = PWM(Pin(28))

framerate = 30
shutter = 6000 # In microseconds

frame_length = 1000000 / framerate
pwm.freq(framerate)

pwm.duty_u16(int((1 - (shutter - 14) / frame_length) * 65535))
```
The low pulsewidth is equal to the shutter time, and the frequency of the PWM equals the framerate.

=== Operation

Run the code on the Pico, and set the camera running:
[,bash]
----
libcamera-hello -t 0 --qt-preview --shutter 3000
----
A frame should now be generated every time that the Pico pulses the pin. Variable framerate is acceptable, and can be controlled by simply
varying the duration between pulses. No options need to be passed to libcamera-apps to enable external trigger.

=== Note
When running libcamera apps, you will need to specify a fixed shutter duration (the value does not matter).
This will ensure the AGC does not try adjusting camera's shutter speed, which is controlled by the external trigger pulse.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
108 changes: 108 additions & 0 deletions documentation/asciidoc/computers/camera/synchronous_cameras.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
== Synchronous Captures

Both the HQ Camera and the Global Shutter Camera, have support for synchronous captures.
Making use of the XVS pin (Vertical Sync) allows one camera to pulse when a frame capture is initiated.
The other camera can then listen for this sync pulse, and capture a frame at the same time as the other camera.

For correct operation, both cameras require a 1.65v pull up voltage on the XVS line, which is created by a potential divider through the 3.3v and GND pins on the Raspberry Pi.
How to setup the potential divider is displayed below.

image::images/synchronous_camera_wiring.jpg[Image showing potential divider setup]

NOTE: Soldering wires to your camera module will void the camera's warranty.

=== HQ Camera
==== Preparation

Connect a potential divider of 2 equally high impedance ( > 1kOhm) resistors to 3v3 and ground, creating 1.65V. This can be connected to either Pi.

Solder the GND and XVS touchpoints of each HQ Camera board to each other.

Connect the XVS wires to the 1.65V potential divider pull up.

==== Boot up both Raspberry Pis

The file `/sys/module/imx477/parameters/trigger_mode` determines which board outputs pulses, or waits to recieve pulses (source and sink).
This parameter can only be altered in superuser mode.

On the sink, run:
[,bash]
----
sudo su
echo 2 > /sys/module/imx477/parameters/trigger_mode
exit
----

On the source, run:
[,bash]
----
sudo su
echo 1 > /sys/module/imx477/parameters/trigger_mode
exit
----

Start the sink running:
[,bash]
----
libcamera-vid --frames 300 --qt-preview -o sink.h264
----

Start the source running
[,bash]
----
libcamera-vid --frames 300 --qt-preview -o source.h264
----

Frames should be synchronous. Use --frames to ensure the same number of frames are captured, and that the recordings are exactly the same length.
Running the sink first ensures that no frames are missed.

==== Note

The potential divider is needed to pull up the XVS pin to high whilst the source is in an idle state.
This ensures that no frames are created or lost upon startup. The source whilst initialising goes from LOW to HIGH which can trigger a false frame.

=== Global Shutter Camera
The global shutter camera can also be operated in a synchonous mode. However, the source will record one extra frame (see xref:camera_software.adoc#note-2[note] below).

A much better alternative method to ensure that both cameras capture the same amount of frames is to use the external trigger method,
described xref:camera_software.adoc#external-trigger-on-the-global-shutter-camera[here].

To operate as source and sink together, the Global Shutter Cameras also require connection of the XHS (horizontal sync) pins together.
However, these do not need connection to a pullup resistor.

==== Preparation

Create a potential divider made of 2x 1.5KOhm Resistors to 3v3 and ground, creating 1.65V. This can be connected to either pi.

Solder 2 wires to the XVS touchpoint on each board and connect both of these wires together to the 1.65V potential divider.

Solder the GND of each Camera board to each other. Also solder 2 wires to the XHS touchpoints on each board and connect these. No pullup is needed for XHS pin.

On the boards that you wish to act as sinks, solder the two halves of the MAS pad together. This enters the sensor into sink mode.

==== Boot up both Raspberry Pis

Start the sink running:
[,bash]
----
libcamera-vid --frames 300 -o sync.h264
----
Allow a delay before you start the source running (see note below). Needs to be roughly > 2 seconds.

Start the source running:
[,bash]
----
libcamera-vid --frames 299 -o sync.h264
----
==== Note
Due to limitations of the IMX296 sensor, we are unable to get the sink to record exactly the same amount of frames as the source.
**The source will record one extra frame before the sink starts recording.** This will need to be accounted for later in the application.
Because of this, you need to specify that the sink records one less frame in the '--frames' option.

FFmpeg has the ability to resync these two videos. By dropping the first frame from the source, we then get two recordings of the same frame
length and with the same starting point.

[,bash]
----
ffmpeg -i source.h264 -vf select="gte(n\, 1)" source.h264
----
4 changes: 4 additions & 0 deletions documentation/asciidoc/computers/camera_software.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,7 @@ include::camera/v4l2.adoc[]
include::camera/csi-2-usage.adoc[]

include::camera/raspicam.adoc[]

include::camera/synchronous_cameras.adoc[]

include::camera/external_trigger.adoc[]

0 comments on commit 100920b

Please sign in to comment.