Difference in RDN upscaled images between browser and Node

[thekevinscott]

Describe the bug
RDN models produce different images between the browser and Node

Screenshots
Div2K 3x, Browser:

Div2K 3x, Node;

[thekevinscott]

I think this may be related to this: tensorflow/tfjs#6627

[thekevinscott]

This is not, in fact, related to tensorflow/tfjs#6627. The fix for this issue is primarily here, though also scattered across a few PRs as well.

The issue here is the behavior of the two main models, the RDN model and the RRDN (GANS) model:

• RRDN (GANS) produces output in the range of [0,1], which then gets post-processed into the range [0,255].
• RDN produces output roughly in the range of [0,255].

I say roughly because sometimes the RDN model can produce output < 0 or > 255, which has unpredictable consequences.

I was relying on a package, tensor-as-base64, to do the translation of tensor to a base64 string. By default, this package clipped the output into the range of [0,255], so the model's output never exceeded this range in the browser.

In Node, for whatever reason, this clipping was inaccurate, or perhaps not being done (to be honest, I'm not entirely sure how this package ever worked on Node in the first place, since it relies on ImageData and canvas neither of which is present in Node, but that's a separate discussion). The solution, then, is to clip the output of the RDN model manually, e.g. with this function, which indeed produces the correct clipped output.

Another thing worth thinking about is whether a base64 src output in Node makes sense at all; I opened a discussion for that here.

(Tagging @danwexler as you were following along on the TFJS thread so this may be of interest to you.)