-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Synchronous and external trigger documentation. #3046
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,67 @@ | ||
== External Trigger on the Global Shutter Camera | ||
The global shutter camera can be triggered externally by pulsing the external trigger (denoted on the board as XTR) connection on the board. | ||
Multiple cameras can be connected to the same pulse, allowing for an alternative way to synchronise two cameras. | ||
|
||
The exposure time is equal to the low pulse-width time plus an additional 14.26us. i.e. a low pulse of 10000us leads to an exposure time of 10014.26us. | ||
Framerate is directly controlled by how often you pulse the pin. A PWM frequency of 30Hz will lead to a framerate of 30 frames per second. | ||
|
||
In this example, we will be using a Raspberry Pi Pico to create the pulses that will drive the camera modules. | ||
|
||
image::images/external_trigger.jpg[Image showing pulse format] | ||
|
||
=== Preparation | ||
|
||
WARNING: This modification includes removing an SMD soldered part. You should not attempt this modification unless you feel you are competent to complete it. | ||
When soldering to the Camera board, please remove the plastic back cover to avoid damaging it. | ||
|
||
**If your board has transistor Q2 fitted, shown in blue on the image below, you will need to remove R11 from the board, as shown in red.** | ||
This connects GP0 to XTR and without removal, the camera will not operate in external trigger mode. | ||
The location of the components is displayed below. | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just one small change needed here:
We also want to circle the Q2 in the image below. |
||
image::images/resistor.jpg[Image showing resistor to be removed] | ||
|
||
Next solder a wire to the touchpoint of XTR on the GS Camera board. | ||
Connect these to the Pico - XTR to any pin. Pin 28 is used in this example. Ground does not need to be connected. | ||
|
||
==== Boot up the Raspberry Pi with the camera connected. | ||
lurch marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
Enable external triggering through superuser mode: | ||
[,bash] | ||
---- | ||
sudo su | ||
echo 1 > /sys/module/imx296/parameters/trigger_mode | ||
exit | ||
---- | ||
|
||
==== Raspberry Pi Pico Micropython Code | ||
[,python] | ||
``` | ||
from machine import Pin, PWM | ||
lurch marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
from time import sleep | ||
|
||
pwm = PWM(Pin(28)) | ||
|
||
framerate = 30 | ||
shutter = 6000 # In microseconds | ||
|
||
frame_length = 1000000 / framerate | ||
pwm.freq(framerate) | ||
|
||
pwm.duty_u16(int((1 - (shutter - 14) / frame_length) * 65535)) | ||
``` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. One thing to correct here - I think strictly speaking this calculation might need to be
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also probably should adjust the pulse diagram to show that exposure time = time for the low pulse width + 14.26us |
||
The low pulsewidth is equal to the shutter time, and the frequency of the PWM equals the framerate. | ||
|
||
=== Operation | ||
|
||
Run the code on the Pico, and set the camera running: | ||
[,bash] | ||
---- | ||
libcamera-hello -t 0 --qt-preview --shutter 3000 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So there's no option that you need to give to the libcamera apps to tell them that they're being used in "external trigger" mode? |
||
---- | ||
A frame should now be generated every time that the Pico pulses the pin. Variable framerate is acceptable, and can be controlled by simply | ||
varying the duration between pulses. No options need to be passed to libcamera-apps to enable external trigger. | ||
|
||
=== Note | ||
When running libcamera apps, you will need to specify a fixed shutter duration (the value does not matter). | ||
This will ensure the AGC does not try adjusting camera's shutter speed, which is controlled by the external trigger pulse. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,106 @@ | ||
== Synchronous Captures | ||
|
||
Both the HQ Camera and the Global Shutter Camera, have support for synchronous captures. | ||
Making use of the XVS pin (Vertical Sync) allows one camera to pulse when a frame capture is initiated. | ||
The other camera can then listen for this sync pulse, and capture a frame at the same time as the other camera. | ||
|
||
For correct operation, both cameras require a 1.65v pull up voltage on the XVS line, which is created by a potential divider through the 3.3v and GND pins on the Raspberry Pi. | ||
How to setup the potential divider is displayed below. | ||
|
||
image::images/synchronous_camera_wiring.jpg[Image showing potential divider setup] | ||
|
||
=== HQ Camera | ||
==== Preparation | ||
|
||
Connect a potential divider of 2 equally high impedance ( > 1kOhm) resistors to 3v3 and ground, creating 1.65V. This can be connected to either Pi. | ||
|
||
Solder the GND and XVS touchpoints of each HQ Camera board to each other. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Worth explaining what XVS stands for? |
||
|
||
Connect the XVS wires to the 1.65V potential divider pull up. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it'd be good to have a wiring diagram here. Perhaps something generated using Fritzing. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can't seem to find any sort of generic camera (or the GS camera) on fritzing. Is there an alternative or shall I just say "Connect to XVS pin" on the wiring diagram? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Must admit the only part I can find is the NoIR camera. A labelled wire heading off to the side is fine. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Feels like a simple wiring diagram would be really useful here? |
||
|
||
==== Boot up both Raspberry Pis | ||
|
||
The file `/sys/module/imx477/parameters/trigger_mode` determines which board outputs pulses, or waits to recieve pulses (source and sink). | ||
This parameter can only be altered in superuser mode. | ||
|
||
On the sink, run: | ||
[,bash] | ||
---- | ||
sudo su | ||
echo 2 > /sys/module/imx477/parameters/trigger_mode | ||
exit | ||
---- | ||
|
||
On the source, run: | ||
[,bash] | ||
---- | ||
sudo su | ||
echo 1 > /sys/module/imx477/parameters/trigger_mode | ||
exit | ||
---- | ||
|
||
Start the sink running: | ||
[,bash] | ||
---- | ||
libcamera-vid --frames 300 --qt-preview -o sink.h264 | ||
---- | ||
|
||
Start the source running | ||
[,bash] | ||
---- | ||
libcamera-vid --frames 300 --qt-preview -o source.h264 | ||
---- | ||
|
||
Frames should be synchronous. Use --frames to ensure the same number of frames are captured, and that the recordings are exactly the same length. | ||
Running the sink first ensures that no frames are missed. | ||
|
||
==== Note | ||
|
||
The potential divider is needed to pull up the XVS pin to high whilst the source is in an idle state. | ||
This ensures that no frames are created or lost upon startup. The source whilst initialising goes from LOW to HIGH which can trigger a false frame. | ||
|
||
=== Global Shutter Camera | ||
The global shutter camera can also be operated in a synchonous mode. However, the source will record one extra frame (see xref:camera_software.adoc#note-2[note] below). | ||
|
||
A much better alternative method to ensure that both cameras capture the same amount of frames is to use the external trigger method, | ||
described xref:camera_software.adoc#external-trigger-on-the-global-shutter-camera[here]. | ||
|
||
To operate as source and sink together, the Global Shutter Cameras also require connection of the XHS (horizontal sync) pins together. | ||
However, these do not need connection to a pullup resistor. | ||
|
||
==== Preparation | ||
|
||
Create a potential divider made of 2x 1.5KOhm Resistors to 3v3 and ground, creating 1.65V. This can be connected to either pi. | ||
|
||
Solder 2 wires to the XVS touchpoint on each board and connect both of these wires together to the 1.65V potential divider. | ||
|
||
Solder the GND of each Camera board to each other. Also solder 2 wires to the XHS touchpoints on each board and connect these. No pullup is needed for XHS pin. | ||
|
||
On the boards that you wish to act as sinks, solder the two halves of the MAS pad together. This enters the sensor into sink mode. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I dunno if it might be worth explaining or clarifying why the wiring for this camera is different for the wiring for the other camera? |
||
|
||
==== Boot up both Raspberry Pis | ||
|
||
Start the sink running: | ||
[,bash] | ||
---- | ||
libcamera-vid --frames 300 -o sync.h264 | ||
---- | ||
Allow a delay before you start the source running (see note below). Needs to be roughly > 2 seconds. | ||
|
||
Start the source running: | ||
[,bash] | ||
---- | ||
libcamera-vid --frames 299 -o sync.h264 | ||
---- | ||
==== Note | ||
Due to limitations of the IMX296 sensor, we are unable to get the sink to record exactly the same amount of frames as the source. | ||
**The source will record one extra frame before the sink starts recording.** This will need to be accounted for later in the application. | ||
lurch marked this conversation as resolved.
Show resolved
Hide resolved
|
||
Because of this, you need to specify that the sink records one less frame in the '--frames' option. | ||
|
||
FFmpeg has the ability to resync these two videos. By dropping the first frame from the source, we then get two recordings of the same frame | ||
length and with the same starting point. | ||
|
||
[,bash] | ||
---- | ||
ffmpeg -i source.h264 -vf select="gte(n\, 1)" source.h264 | ||
---- |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -61,3 +61,7 @@ include::camera/v4l2.adoc[] | |
include::camera/csi-2-usage.adoc[] | ||
|
||
include::camera/raspicam.adoc[] | ||
|
||
include::camera/synchronous_cameras.adoc[] | ||
|
||
include::camera/external_trigger.adoc[] | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. At the end seems reasonable at the moment. The legacy camera stack stuff is going away on the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry late to comment on this, but I would remove the warranty sentence completely.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed.
After discussions with @simon-martin-rpi, I'd probably go with something along the lines of "WARNING: The modification includes removing an SMD soldered part. You should not attempt this modification unless you feel you are competent to complete it." or something along those lines.