feat: Add int/bigint/double conversion functions from bytes #8426

spena · 2021-11-29T23:45:39Z

Description

Add INT_FROM_BYTES, BIGINT_FROM_BYTES, and DOUBLE_FROM_BYTES functions.

All functions use the byte order BIG_ENDIAN as default. A 2nd parameter is accepted to specify the byte order:

SELECT INT_FROM_BYTES(b) ...
SELECT INT_FROM_BYTES(b, 'BIG_ENDIAN') ...
SELECT INT_FROM_BYTES(b, 'LITTLE_ENDIAN') ...

Testing done

Describe the testing strategy. Unit and integration tests are expected for any behavior changes.

Reviewer checklist

Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
Ensure relevant issues are linked (description should include text like "Fixes #")

spena · 2021-11-29T23:46:54Z

@jzaralim Instead of the proposed int_from_bytes(b, 'little'), I used int_from_bytes(b, 'little_endian'). What do you think? It maches the current ByteOrder.LITTLE_ENDIAN and also gives a better meaning when looking at the function call.

JimGalasyn

Do we want to add docs to Scalar functions?

jzaralim · 2021-11-30T18:05:37Z

ksqldb-engine/src/main/java/io/confluent/ksql/function/udf/conversions/IntFromBytes.java

+      @UdfParameter(description = "The byte order of the number bytes representation")
+      final String byteOrder
+  ) {
+    if (byteOrder.equalsIgnoreCase(ByteOrder.BIG_ENDIAN.toString())) {


We should do a null check for byteOrder (other two functions as well)

jzaralim · 2021-11-30T18:06:37Z

ksqldb-engine/src/main/java/io/confluent/ksql/function/udf/conversions/IntFromBytes.java

+  public Integer intFromBytes(
+      @UdfParameter(description = "The bytes value to convert.")
+      final ByteBuffer value,
+      @UdfParameter(description = "The byte order of the number bytes representation")


Can we add the names of the byte orders in the description (LITTLE_ENDIAN and BIG_ENDIAN)? (also applies to the other two functions)

spena · 2021-12-01T21:46:57Z

@JimGalasyn I updated the doc

docs/developer-guide/ksqldb-reference/scalar-functions.md

JimGalasyn

LGTM with a few copy edits.

jzaralim

LGTM with a nit

jzaralim · 2021-12-02T01:57:24Z

ksqldb-common/src/main/java/io/confluent/ksql/util/BytesUtils.java

@@ -65,6 +67,18 @@ private BytesUtils() {
      Encoding.BASE64, v -> base64Decoding(v)
  );

+  public static ByteOrder byteOrderType(final String byteOrderStr) {
+    if (byteOrderStr.equalsIgnoreCase(ByteOrder.BIG_ENDIAN.toString())) {


If byteOrderStr is null then byteOrderStr.equalsIgnoreCase will throw an NPE

spena requested review from jzaralim and a team November 29, 2021 23:45

spena requested a review from JimGalasyn as a code owner November 29, 2021 23:45

JimGalasyn reviewed Nov 29, 2021

View reviewed changes

jzaralim reviewed Nov 30, 2021

View reviewed changes

spena force-pushed the feat_numeric_bytes_functions branch from f116957 to 6a2dd34 Compare December 1, 2021 21:46

JimGalasyn reviewed Dec 1, 2021

View reviewed changes