SD Webui Vectorscope CC

This is an Extension for the Automatic1111 Webui, which performs a kind of Offset Noise natively during inference, allowing you to adjust the brightness, contrast, and color of the generations.

Also supports both old & new Forge

Example Images

Base Image w/o Extension

Infotext

• Checkpoint: realisticVisionV51
• Positive Prompt: (high quality, best quality), a 4k cinematic photo of a gentleman in suit, street in a city at night, (depth of field, bokeh)
• Negative Prompt: (low quality, worst quality:1.2), [EasyNegative, EasyNegativeV2]

Steps: 32, Sampler: DPM++ 2M Karras, CFG scale: 7.5, Seed: 3709157017, Size: 512x512, Denoising strength: 0.5
Clip skip: 2, Token merging ratio: 0.2, Token merging ratio hr: 0.2, RNG: CPU, NGMS: 4
Hires upscale: 2, Hires steps: 16, Hires upscaler: 2xNomosUni_esrgan_multijpg

Vibrant	Cold	"Movie when Mexico"
Alt: True Saturation: 1.75 Noise: Ones Scaling: 1 - Cos	Brightness: -5.0 Contrast: 2.5 Saturation: 0.75 R: -3.0 B: 3.0 Noise: Ones Scaling: 1 - Sin	Brightness: 2.5 Contrast: -2.5 Saturation: 1.25 R: 1.5 G: 3.0 B: -4.0 Noise: Ones Scaling: 1 - Sin

How to Use

Note: Since this Extension modifies the underlying latent tensor, the composition may change drastically depending on the parameters

Basic Parameters

• Enable: Enable the Extension 💀

• Alt: Cause the Extension effects to be stronger

  Technical Detail

  • This parameter makes the Extension modify the denoised Tensor instead of the x Tensor

• Brightness, Contrast, Saturation: Adjust the overall brightness / contrast / saturation of the image

Color Channels

Channel	Lower	Higher
R	Cyan	Red
G	Magenta	Green
B	Yellow	Blue

• The Extension also comes with a Color Wheel for visualization, which you can also click on to pick a color directly

The color picker isn't 100% accurate due to multiple layers of conversions...

Style Presets

• To apply a Style, select from the Dropdown then click Apply Style
• To save a Style, enter a name in the Textbox then click Save Style
• To delete a Style, enter the name in the Textbox then click Delete Style
  • Style that was deleted is still in the styles.json in case you wish to retrieve it
• Click Refresh Style to update the Dropdown if you edited the styles.json manually

You can also find pre-made Styles by the community available online

• eg. The Photomatix Styles (right click on the link, click Save link as, then save the .json file into the sd-webui-vectorscope-cc extension folder)

Advanced Parameters

• Process Hires. fix: Enable this option to process during the Hires. fix phase too
  • By default, this Extension only functions during the regular phase of the txt2img mode
• Process ADetailer: Enable this option to process during the ADetailer phase too
  • Will usually cause a square of inconsistent colors
• Randomize using Seed: Enable this option to use the current generation seed to randomize the basic parameters
  • Randomized results will be printed in the console

Noise Settings

let x denote the latent Tensor ; let y denote the operations

• Straight: All operations are calculated on the same Tensor
  • x += x * y
• Cross: All operations are calculated on the Tensor opposite of the Alt. setting
  • x += x' * y
• Ones: All operations are calculated on a Tensor filled with ones
  • x += 1 * y
• N.Random: All operations are calculated on a Tensor filled with random values in normal distribution
  • x += randn() * y
• U.Random: All operations are calculated on a Tensor filled with random values in uniform distribution
  • x += rand() * y
• Multi-Res: All operations are calculated on a Tensor generated with multi-res noise algorithm
  • x += multires() * y
• Abs: Calculate using the absolute values of the chosen Tensors instead
  • x += abs(F) * y

Infotext

• Checkpoint: realisticVisionV51
• Positive Prompt: (high quality, best quality), a 4k photo of a cute dog running in the snow, mountains, day, (depth of field, bokeh)
• Negative Prompt: (low quality, worst quality:1.2), [EasyNegative, EasyNegativeV2]
• Brightness: 2.5
• Contrast: 2.5
• Alt: True
• Scaling: 1 - Cos

Steps: 24, Sampler: DPM++ 2M Karras, CFG scale: 7.5, Seed: 1257068736, Size: 512x512, Denoising strength: 0.5
Clip skip: 2, Token merging ratio: 0.2, Token merging ratio hr: 0.2, RNG: CPU, NGMS: 4
Hires upscale: 1.5, Hires steps: 16, Hires upscaler: 2xNomosUni_esrgan_multijpg

Scaling Settings

By default, this Extension offsets the noise by the same amount every step. But depending on the Sampler and Scheduler used, and whether Alt. was enabled or not, the effects might be too strong during the early or the later phase of the process, which in turn causes artifacts.

• Flat: Default behavior
• Cos: Cosine scaling (High -> Low)
• Sin: Sine scaling (Low -> High)
• 1 - Cos: (Low -> High)
• 1 - Sin: (High -> Low)

Infotext

• Checkpoint: realisticVisionV51
• Positive Prompt: (high quality, best quality), a 4k photo of a cute cat standing at a flower field in a park, day, (depth of field, bokeh)
• Negative Prompt: (low quality, worst quality:1.2), [EasyNegative, EasyNegativeV2]
• Alt: True
• Noise: Straight Abs.

Steps: 24, Sampler: DPM++ 2M Karras, CFG scale: 7.5, Seed: 3515074713, Size: 512x512, Denoising strength: 0.5
Clip skip: 2, Token merging ratio: 0.2, Token merging ratio hr: 0.2, RNG: CPU, NGMS: 4
Hires upscale: 1.5, Hires steps: 12, Hires upscaler: 2xNomosUni_esrgan_multijpg

Buttons

• Reset: Reset all Basic and Advanced parameters to the default values
• Randomize: Randomize the Brightness, Contrast, Saturation, R, G, B parameters

Roadmap

• Extension Released!
• Add Support for X/Y/Z Plot
• Implement different Noise functions
• Implement Randomize button
• Implement Style Presets
• Implement Color Wheel & Color Picker
• Implement better scaling algorithms
• Add API Docs
• Append Parameters to Infotext
• Improved Infotext Support (by. catboxanon)
• Add HDR Script
• Add Support for SDXL
• Implement Gradient features

API

You can use this Extension via API by adding an entry to the alwayson_scripts of your payload. An example is provided. The args are sent in the following order in an array:

Parameter	Type
Enable	bool
Alt.	bool
Brightness	float
Contrast	float
Saturation	float
R	float
G	float
B	float
Hires. fix	bool
ADetailer	bool
Randomize	bool
Noise Method	str
Scaling	str

Known Issues

• In rare occasions, this Extension has little effects when used with certain LoRAs
• Works better / worse with certain Samplers

HDR

Discussion Thread

In the Script Dropdown at the bottom, there is now a new High Dynamic Range option:

• This script will generate multiple images ("Brackets") of varying brightness, then merge them into 1 HDR image
• (Recommended) Use a deterministic sampler and high enough steps. Euler (not Euler a) works well in my experience

Settings

• Brackets: The numer of images to generate
• Gaps: The brightness difference between each image
• Automatically Merge: When enabled, this will merge the images using an OpenCV algorithm and save to the HDR folder in the outputs folder
  • Disable this if you want to merge them yourself using better external program

---

Offset Noise TL;DR

The most common version of Offset Noise you may have heard of is from this blog post, where it was discovered that the noise functions used during training were flawed, causing Stable Diffusion to always generate images with an average of 0.5 (ie. grey).

ie. Even if you prompt for dark/night or bright/snow, the average of the image is still "grey"

Technical Explanations

However, this Extension instead tries to offset the latent noise during the inference phase. Therefore, you do not need to use models that were specially trained, as this can work on any model.

How does this work?

After reading through and messing around with the code, I found out that it is possible to directly modify the Tensors representing the latent noise used by the Stable Diffusion process.

The dimensions of the Tensors is (X, 4, H / 8, W / 8), which represents X batch of noise images, with 4 channels, each with (W / 8) x (H / 8) values

eg. Generating a single 512x768 image will create a Tensor of size (1, 4, 96, 64)

Then, I tried to play around with the values of each channel and ended up discovering these relationships. Essentially, the 4 channels correspond to the CMYK color format for SD1 (Y'CbCr for SDXL), hence why you can control the brightness as well as the colors.

---

Vectorscope?

The Extension is named this way because the color interactions remind me of the Vectorscope found in Premiere Pro's Lumetri Color. Those who are experienced in Color Correction should be rather familiar with this Extension.

^{Yes. I'm aware that it's just how digital colors work in general.}

^{We've come full circle (*ba dum tss) now that a Color Wheel is actually added.}