Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Synchronous and external trigger documentation. #3046

Merged
merged 1 commit into from
Aug 23, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions documentation/asciidoc/computers/camera/external_trigger.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
== External Trigger on the Global Shutter Camera
The global shutter camera can be triggered externally by pulsing the external trigger (denoted on the board as XTR) connection on the board.
Multiple cameras can be connected to the same pulse, allowing for an alternative way to synchronise two cameras.

The exposure time is equal to the low pulse-width time plus an additional 14.26us. i.e. a low pulse of 10000us leads to an exposure time of 10014.26us.
Framerate is directly controlled by how often you pulse the pin. A PWM frequency of 30Hz will lead to a framerate of 30 frames per second.

In this example, we will be using a Raspberry Pi Pico to create the pulses that will drive the camera modules.

image::images/external_trigger.jpg[Image showing pulse format]

=== Preparation

WARNING: This modification includes removing an SMD soldered part. You should not attempt this modification unless you feel you are competent to complete it.
When soldering to the Camera board, please remove the plastic back cover to avoid damaging it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry late to comment on this, but I would remove the warranty sentence completely.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed.

After discussions with @simon-martin-rpi, I'd probably go with something along the lines of "WARNING: The modification includes removing an SMD soldered part. You should not attempt this modification unless you feel you are competent to complete it." or something along those lines.

**If your board has transistor Q2 fitted, shown in blue on the image below, you will need to remove R11 from the board, as shown in red.**
This connects GP0 to XTR and without removal, the camera will not operate in external trigger mode.
The location of the components is displayed below.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one small change needed here:

First, remove/desolder resistor 11 on the GS Camera board. This connects GP0 to XTR and without removal, the camera will not operate in external trigger mode. If you have transistor Q2 fitted, remove/desolder the component along with R11. This is needed as the drive strength on the Raspberry Pi Pico is not strong enough to bring the XTRIG line low, and the the camera will not operate in external trigger mode.

We also want to circle the Q2 in the image below.

image::images/resistor.jpg[Image showing resistor to be removed]

Next solder a wire to the touchpoint of XTR on the GS Camera board.
Connect these to the Pico - XTR to any pin. Pin 28 is used in this example. Ground does not need to be connected.

==== Boot up the Raspberry Pi with the camera connected.
lurch marked this conversation as resolved.
Show resolved Hide resolved

Enable external triggering through superuser mode:
[,bash]
----
sudo su
echo 1 > /sys/module/imx296/parameters/trigger_mode
exit
----

==== Raspberry Pi Pico Micropython Code
[,python]
```
from machine import Pin, PWM
lurch marked this conversation as resolved.
Show resolved Hide resolved

from time import sleep

pwm = PWM(Pin(28))

framerate = 30
shutter = 6000 # In microseconds

frame_length = 1000000 / framerate
pwm.freq(framerate)

pwm.duty_u16(int((1 - (shutter - 14) / frame_length) * 65535))
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing to correct here - I think strictly speaking this calculation might need to be

pwm.duty_u16(int((1 - (shutter - 14) / frame_length) * 65535))

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also probably should adjust the pulse diagram to show that exposure time = time for the low pulse width + 14.26us

The low pulsewidth is equal to the shutter time, and the frequency of the PWM equals the framerate.

=== Operation

Run the code on the Pico, and set the camera running:
[,bash]
----
libcamera-hello -t 0 --qt-preview --shutter 3000
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So there's no option that you need to give to the libcamera apps to tell them that they're being used in "external trigger" mode?

----
A frame should now be generated every time that the Pico pulses the pin. Variable framerate is acceptable, and can be controlled by simply
varying the duration between pulses. No options need to be passed to libcamera-apps to enable external trigger.

=== Note
When running libcamera apps, you will need to specify a fixed shutter duration (the value does not matter).
This will ensure the AGC does not try adjusting camera's shutter speed, which is controlled by the external trigger pulse.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
106 changes: 106 additions & 0 deletions documentation/asciidoc/computers/camera/synchronous_cameras.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
== Synchronous Captures

Both the HQ Camera and the Global Shutter Camera, have support for synchronous captures.
Making use of the XVS pin (Vertical Sync) allows one camera to pulse when a frame capture is initiated.
The other camera can then listen for this sync pulse, and capture a frame at the same time as the other camera.

For correct operation, both cameras require a 1.65v pull up voltage on the XVS line, which is created by a potential divider through the 3.3v and GND pins on the Raspberry Pi.
How to setup the potential divider is displayed below.

image::images/synchronous_camera_wiring.jpg[Image showing potential divider setup]

=== HQ Camera
==== Preparation

Connect a potential divider of 2 equally high impedance ( > 1kOhm) resistors to 3v3 and ground, creating 1.65V. This can be connected to either Pi.

Solder the GND and XVS touchpoints of each HQ Camera board to each other.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth explaining what XVS stands for?


Connect the XVS wires to the 1.65V potential divider pull up.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it'd be good to have a wiring diagram here. Perhaps something generated using Fritzing.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't seem to find any sort of generic camera (or the GS camera) on fritzing. Is there an alternative or shall I just say "Connect to XVS pin" on the wiring diagram?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't seem to find any sort of generic camera (or the GS camera) on fritzing. Is there an alternative or shall I just say "Connect to XVS pin" on the wiring diagram?

Must admit the only part I can find is the NoIR camera. A labelled wire heading off to the side is fine.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feels like a simple wiring diagram would be really useful here?


==== Boot up both Raspberry Pis

The file `/sys/module/imx477/parameters/trigger_mode` determines which board outputs pulses, or waits to recieve pulses (source and sink).
This parameter can only be altered in superuser mode.

On the sink, run:
[,bash]
----
sudo su
echo 2 > /sys/module/imx477/parameters/trigger_mode
exit
----

On the source, run:
[,bash]
----
sudo su
echo 1 > /sys/module/imx477/parameters/trigger_mode
exit
----

Start the sink running:
[,bash]
----
libcamera-vid --frames 300 --qt-preview -o sink.h264
----

Start the source running
[,bash]
----
libcamera-vid --frames 300 --qt-preview -o source.h264
----

Frames should be synchronous. Use --frames to ensure the same number of frames are captured, and that the recordings are exactly the same length.
Running the sink first ensures that no frames are missed.

==== Note

The potential divider is needed to pull up the XVS pin to high whilst the source is in an idle state.
This ensures that no frames are created or lost upon startup. The source whilst initialising goes from LOW to HIGH which can trigger a false frame.

=== Global Shutter Camera
The global shutter camera can also be operated in a synchonous mode. However, the source will record one extra frame (see xref:camera_software.adoc#note-2[note] below).

A much better alternative method to ensure that both cameras capture the same amount of frames is to use the external trigger method,
described xref:camera_software.adoc#external-trigger-on-the-global-shutter-camera[here].

To operate as source and sink together, the Global Shutter Cameras also require connection of the XHS (horizontal sync) pins together.
However, these do not need connection to a pullup resistor.

==== Preparation

Create a potential divider made of 2x 1.5KOhm Resistors to 3v3 and ground, creating 1.65V. This can be connected to either pi.

Solder 2 wires to the XVS touchpoint on each board and connect both of these wires together to the 1.65V potential divider.

Solder the GND of each Camera board to each other. Also solder 2 wires to the XHS touchpoints on each board and connect these. No pullup is needed for XHS pin.

On the boards that you wish to act as sinks, solder the two halves of the MAS pad together. This enters the sensor into sink mode.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dunno if it might be worth explaining or clarifying why the wiring for this camera is different for the wiring for the other camera?


==== Boot up both Raspberry Pis

Start the sink running:
[,bash]
----
libcamera-vid --frames 300 -o sync.h264
----
Allow a delay before you start the source running (see note below). Needs to be roughly > 2 seconds.

Start the source running:
[,bash]
----
libcamera-vid --frames 299 -o sync.h264
----
==== Note
Due to limitations of the IMX296 sensor, we are unable to get the sink to record exactly the same amount of frames as the source.
**The source will record one extra frame before the sink starts recording.** This will need to be accounted for later in the application.
lurch marked this conversation as resolved.
Show resolved Hide resolved
Because of this, you need to specify that the sink records one less frame in the '--frames' option.

FFmpeg has the ability to resync these two videos. By dropping the first frame from the source, we then get two recordings of the same frame
length and with the same starting point.

[,bash]
----
ffmpeg -i source.h264 -vf select="gte(n\, 1)" source.h264
----
4 changes: 4 additions & 0 deletions documentation/asciidoc/computers/camera_software.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,7 @@ include::camera/v4l2.adoc[]
include::camera/csi-2-usage.adoc[]

include::camera/raspicam.adoc[]

include::camera/synchronous_cameras.adoc[]

include::camera/external_trigger.adoc[]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about the ordering here. Perhaps @aallan or @lurch might be able to advise?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the end seems reasonable at the moment. The legacy camera stack stuff is going away on the bookworm branch so as far as I remember the entire raspicam.adoc section disappears.

Loading