Kotlin serialization migration #190

Derek52 · 2021-04-10T03:29:34Z

This pr allows for de/serialization of Server2ClientMessage with Kotlin Serialization. You've seen some of this code before in the previous pr that was closed, but it was harder to find due to so many redundant commits. This should be easier to follow.

Also, I've added a TODO in misc.kt, because while I think Client2ServerMessage should be straightforward to move over, misc.kt has Any.toJson(), and I think that might be a little more involved to fix.

We could do something like the primitiveToJson() function I added to that file.
Objects that are marked @serializable would probably also work if we rewrote it to use Kotlin Serialization.
Ideally, we'd be able to do something like
when (this) {
is serializable -> Json.encodeToString(this)
else -> error("You tried to encode a non serializable class")
}
But, I did a quick search, and I couldn't find a way to check if a class is Serializable, programmatically.

Alternatively, I thought instead of rewriting this Any.toJson(), we could replace usages of it, with something like primitiveToJson()
So, I looked for project usages of this function and randomly selected this one, from StorageReceiver.kt
operator fun set(name: String, value: Any) {
setString(name, value.toJson())
}

It seems like limiting that function to primitive types might work ok. But, we might want to support non primitive types for other function calls.. I haven't gone through every usage of Any.toJson() yet to see if using only primitives would cause problems. It's probably better to support objects too. I just don't see a great way to do that yet.

# Conflicts: # .gitignore # build.gradle # src/main/kotlin/kweb/Element.kt # src/main/kotlin/kweb/html/Document.kt

…ests now pass.

…n_Serialization_Migration # Conflicts: # build.gradle # src/main/kotlin/kweb/Element.kt # src/main/kotlin/kweb/ElementCreator.kt # src/main/kotlin/kweb/WebBrowser.kt # src/main/kotlin/kweb/client/Server2ClientMessage.kt # src/main/kotlin/kweb/html/fileUpload/FileFormInput.kt # src/main/kotlin/kweb/util/misc.kt

…fail.

sanity · 2021-04-11T17:04:10Z

Re replacing Any.toJson(), would this be a solution: https://kotlin.github.io/kotlinx.serialization/kotlinx-serialization-json/kotlinx-serialization-json/kotlinx.serialization.json/encode-to-json-element.html

src/main/kotlin/kweb/Kweb.kt

src/main/kotlin/kweb/WebBrowser.kt

src/main/kotlin/kweb/util/misc.kt

Derek52 · 2021-04-23T03:05:50Z

This branch is a mess at the moment. There are conflicts with master, and It doesn't even compile at the moment. But, I wanted you to see what it looks like, to make sure we'll be happy going this direction. Also, I'm having an issue with 2 functions in Element.kt

These are setAttributeRaw() and setAttribute(). I'm not sure how far up the ladder I should be changing Any to JsonElement. Fixing setAttributeRaw() could easily be fixed by making this change. I could also try and have setAttributeRaw() call another function, that is overloaded to support each primitive type, like we did for set() in CookieReceiver.kt

fun setAttribute(name: String, oValue: KVal): Element {
setAttributeRaw(name, oValue.value)
val handle = oValue.addListener { _, newValue ->
setAttributeRaw(name, newValue)
}
this.creator?.onCleanup(true) {
oValue.removeListener(handle)
}
return this
}

Fixing setAttribute() is a little trickier though. If I overload setAttributeRaw(), we might not even need to change setAttribute(). But, if we make setAttributeRaw() take a JsonElement, than setAttribute will have to convert oValue to JsonElement. And it can't do that. Unless, we rewrite KVal to use JsonElement instead of a generic. Which, seems like it could be appealing. I don't know how much work that would take, or if it is a direction we want to go. But, I think it has potential.

src/main/kotlin/kweb/CookieReceiver.kt

src/main/kotlin/kweb/Element.kt

src/main/kotlin/kweb/Kweb.kt

src/main/kotlin/kweb/html/StorageReceiver.kt

src/main/kotlin/kweb/util/misc.kt

sanity · 2021-04-23T14:20:04Z

These are setAttributeRaw() and setAttribute(). I'm not sure how far up the ladder I should be changing Any to JsonElement. Fixing setAttributeRaw() could easily be fixed by making this change. I could also try and have setAttributeRaw() call another function, that is overloaded to support each primitive type, like we did for set() in CookieReceiver.kt

I think both setAttribute() and setAttributeRaw() should take a JsonPrimitive as a value (or a KVal in the case of setAttribute()), since in practice this is what an attribute value will be (a JavaScript primitive).

If we do this then I think it means we can get rid of setAttributeRaw() and just rely on overloading to decide whether the value is a KVal<JsonPrimitive> or a JsonPrimitive. The reason I couldn't do this before is because the value was an Any which also matched when we gave it a KVal.

So, to be explicit, the new functions would be:

   fun setAttribute(name : String, value : JsonPrimitive) { .. }

  fun setAttribute(name : String, value : KVal<JsonPrimitive>) { .. }

Fixing setAttribute() is a little trickier though. If I overload setAttributeRaw(), we might not even need to change setAttribute(). But, if we make setAttributeRaw() take a JsonElement, than setAttribute will have to convert oValue to JsonElement. And it can't do that. Unless, we rewrite KVal to use JsonElement instead of a generic. Which, seems like it could be appealing. I don't know how much work that would take, or if it is a direction we want to go. But, I think it has potential.

I'm not sure I understand the issue here, is it solved if we just decide that the value must be a JsonPrimitive as discussed above?

…tions from misc.kt

Derek52 · 2021-04-25T02:21:23Z

The issue I was having is resolved by using JsonPrimitive like in your example there. I am wondering though if we should rewrite Kval to have a JsonElement as a value, instead of using the Template. The template is starting to feel a little unnecessary to me. I could be wrong though.

Also, my 2 commits here, have gotten me into a much better place. Stuff is still broken though. Kweb.kt's respond function is now erroring with "Expected Begin Object but got String", which is the opposite of what it used to error with. I'm sure I can fix that with a bit more modification. I've also run the todo app. The event still isn't working with the add item button. And, the page no longer renders correctly. It looks like it is missing some styling. Let me know if anything in the commits jumps out at you that would cause that to happen. It's probably pretty simple.

sanity · 2021-04-25T03:36:16Z

I am wondering though if we should rewrite Kval to have a JsonElement as a value, instead of using the Template

I'm not sure I understand what you mean by "Template" in this context, could you elaborate?

Let me know if anything in the commits jumps out at you that would cause that to happen. It's probably pretty simple.

I'll take a look.

sanity · 2021-04-25T03:37:56Z

src/main/kotlin/kweb/CookieReceiver.kt

@@ -27,7 +65,7 @@ class CookieReceiver(val receiver: WebBrowser) {
            arguments.add(domain)
        }

-        receiver.callJsFunction("docCookies.setItem({});", arguments.joinToString(separator = ", "))
+        receiver.callJsFunction("docCookies.setItem({});", JsonPrimitive(arguments.joinToString(separator = ", ")))


Not sure, is this definitely what setItem() is expecting (a string containing a comma-separated list)?

No it is not. This is a mistake I made awhile ago and it slipped through the cracks. It's an easy fix though.

Derek52 · 2021-04-26T00:40:44Z

Sorry, "Template" was the wrong word. I think I should have said "Generic". Template is a C++ word.

But, what I'm saying is, maybe Kval should go from

open class KVal<T : Any?>(value: T)

to

open class KVal(value: JsonElement)

sanity · 2021-04-26T13:43:17Z

Hmm, I'm not sure I see the advantage of that and it would significantly reduce the utility of KVals and KVars. In particular, when using a KVar/KVal with render() { } it's important to be able to use types other than JsonPrimitive.

Derek52 · 2021-04-26T17:24:39Z

I said JsonElement, not JsonPrimitive. The idea is that for functions like setAttribute(), we could do value : KVal instead of value : KVal<JsonPrimitive>. It seemed like a viable idea since we are moving more and more code to using JsonElement. I hadn't thought about this too deeply though. I was overlooking things like render(), and I agree that it probably isn't worth it.

…ient-side.

…ent side.

Derek52 · 2021-05-07T02:14:02Z

We have 4 uses of gson.fromJson() that are tripping me up. I'm not 100% sure how they work and what they do, and I don't want to accidentally break something by changing them incorrectly. These 4 uses are in

OnReceiver
https://github.com/Derek52/kweb-core/blob/8ee58f9012990f9702b803931245ae4c91f4875c/src/main/kotlin/kweb/html/events/OnReceiver.kt#L24-L35

CookieReceiver
https://github.com/Derek52/kweb-core/blob/8ee58f9012990f9702b803931245ae4c91f4875c/src/main/kotlin/kweb/CookieReceiver.kt#L69-L75

StorageReceiver
https://github.com/Derek52/kweb-core/blob/8ee58f9012990f9702b803931245ae4c91f4875c/src/main/kotlin/kweb/html/StorageReceiver.kt#L64-L71

And JqueryCore events
https://github.com/Derek52/kweb-core/blob/8ee58f9012990f9702b803931245ae4c91f4875c/src/main/kotlin/kweb/plugins/jqueryCore/jqueryevents.kt#L26-L34

The last one is a plugin, and per a previous discussion, we could probably just remove that one. For CookieReceiver and StorageReceiver I rewrote the function to look like this,

suspend inline fun get(name: String) : JsonElement {
        val result = getString(name)
        return Json.encodeToJsonElement(name)
}

That doesn't break anything in the tests, or either demo app I've been using. But, I'm not convinced it will work in as many places as the old version. And I'm less clear on how to handle the code block in OnReceiver I'm not clear where the type of propertiesAsString is declared, and how we can use the correct Kotlin Serializer on it.

…o 1.2.0, because there seems to be a bug there.

…xtra quotes being added to Strings from the client.

…ion_Migration # Conflicts: # build.gradle # src/main/kotlin/kweb/Element.kt # src/main/kotlin/kweb/Kweb.kt

sanity

Looks great, just some minor nitpicks.

sanity · 2021-05-13T14:11:25Z

build.gradle

@@ -45,17 +45,18 @@ dependencies {
    compile 'org.apache.commons:commons-lang3:3.11'
    compile 'commons-io:commons-io:2.8.0'
    compile 'org.jsoup:jsoup:1.13.1'
-    compile 'com.github.kwebio:shoebox:0.4.12'
+    compile 'com.github.kwebio:shoebox:0.4.5'


Current shoebox version is 0.4.35 (I had to do a bunch of releases while trying to fix that problem :( ).

This was a typo. I'm surprised Kweb still built and ran when asking for a non existent version of Shoebox.

sanity · 2021-05-13T14:12:23Z

build.gradle

    compile 'org.jetbrains.kotlinx:kotlinx-serialization-json:1.1.0-RC'
+    //implementation 'org.jetbrains.kotlin:kotlin-reflect:1.5.0'


Nitpick but should probably remove dead code.

sanity · 2021-05-13T14:13:06Z

src/main/kotlin/kweb/CookieReceiver.kt

        }
    }

+    /*suspend inline fun <reified V : Any> get(name: String): V? {


Should probably remove dead code.

sanity · 2021-05-13T14:18:04Z

src/main/kotlin/kweb/Element.kt

        """.trimIndent()
+        //Adding event listener was causing the client to send a Client2ServerMessage with a null data field. This caused an error
+        //We make the client return true to avoid that issue.
+        //Then on the server we only invoke our callback on eventObjects, by checking that payload is a JsonObject.


Might want to add a TODO to find a way to avoid the initial ignored message.

sanity · 2021-05-13T14:22:33Z

src/main/kotlin/kweb/html/StorageReceiver.kt

+        setJson(name, value)
+    }
+
+    fun setJson(key: String, value: JsonElement) {


should we just merge this into fun set(name: String, value: JsonElement) since they do the same thing?

Yes, we should probably merge them.

Also, I'm now realizing we decided to write the class this way, when we were still abstracting away JsonElements and stuff from end users. We could have fun set(key: String, value: JsonElement), and just have end users convert their data to JsonPrimitive(or object or element), instead of us doing it for them. That'd remove a lot of duplication in this class, and in CookieReceiver

I think those convenience functions for JsonPrimitives are useful, they do lead to some duplication but I think its ok in this case.

sanity · 2021-05-13T14:22:48Z

src/main/kotlin/kweb/html/StorageReceiver.kt

        }
    }

+    /*suspend inline fun <reified V : Any> get(name: String): V? {


Should probably remove dead code.

…ion_Migration

Derek52 and others added 13 commits March 15, 2021 12:15

Attempt to migrate Gson to Kotlin Serialization

a9f8095

Attempt to use JsonPrimitive

17f601e

Merge branch 'master' into Issue-59-migrate-from-gson-to-kotlin-serial

c97d34d

# Conflicts: # .gitignore # build.gradle # src/main/kotlin/kweb/Element.kt # src/main/kotlin/kweb/html/Document.kt

Merge changes to Document.kt & Element.kt from Master

790d1d3

Convert Server2ClientMessage to use JsonElement instead of Any. All t…

ae89ed1

…ests now pass.

Refactor and streamline code to convert callJs arguments to JsonElement

5514d3b

Refactor code supporting hashmaps being passed as JS function arguments.

ad08461

Fix an issue with reflection not working.

83240e2

Fix redundant string template that was causing ImmediateEventTest to …

f301f41

…fail.

Clean up code and add TODO.

2267064

Fix issue with empty javascript message.

dd84dd7

Clean up a comment.

cf56855

sanity reviewed Apr 11, 2021

View reviewed changes

src/main/kotlin/kweb/Kweb.kt Show resolved Hide resolved

src/main/kotlin/kweb/WebBrowser.kt Outdated Show resolved Hide resolved

src/main/kotlin/kweb/util/misc.kt Outdated Show resolved Hide resolved

Use encodeToJsonElement()

9641b5e

sanity mentioned this pull request Apr 17, 2021

Memory leaks due to un-GCable event handlers #194

Closed

sanity marked this pull request as draft April 20, 2021 13:05

Derek52 added 3 commits April 22, 2021 16:40

Mark Client2ServerMessage.kt serializable.

c774ac8

Remove misc.kt Any.toJson() and fix breaking code.

2e00a87

Convert callJsFunction calls to use JsonPrimitive.

5609324

sanity suggested changes Apr 23, 2021

View reviewed changes

Derek52 added 2 commits April 24, 2021 21:14

Convert more calls to use JsonPrimitive instead of toJson().

7b0548a

Modify Cookie and Storage receiver to remove .toJson() extension func…

50becdf

…tions from misc.kt

sanity suggested changes Apr 25, 2021

View reviewed changes

Derek52 added 6 commits April 27, 2021 17:07

Fix call to docCookies.setItem() in CookieReceiver.kt

2dde58e

Move more code from Any to JsonElement.

23029fe

Remove dead code.

4f6d665

Fix issue with html attributes receiving redundant quote marks.

5e423bd

Fix issue with HTML class names not being assigned properly on the cl…

8ee58f9

…ient-side.

Fix issue that caused OnImmediateReceiver.kt demo to error on the cli…

56cf55e

…ent side.

Derek52 added 13 commits May 7, 2021 22:10

Upgrade Kotlin library version. Do not upgrade Kotlin Serialization t…

32b9067

…o 1.2.0, because there seems to be a bug there.

Fix issue that prevented Events from running, and clean up code.

a5f0503

Update Shoebox dependency version

56faf30

Make cacheId a number instead of String.

fa15511

Migrate callbacks to use JsonElement instead of Any. Fix issue with e…

d0298fa

…xtra quotes being added to Strings from the client.

Optimize imports and fix redundant JS in Server2ClientMessage

f445f6e

Clean up get_diff_changes() in bootstrap.js

7be09db

Fix bug with Client2Server sending null data.

12f34ae

Fix issue with Kvar's not updating.

be0b8e0

Bump Kweb version to 0.9.0

167586e

Remove Gson.

c24e58b

Merge remote-tracking branch 'upstream/master' into Kotlin_Serializat…

e283c27

…ion_Migration # Conflicts: # build.gradle # src/main/kotlin/kweb/Element.kt # src/main/kotlin/kweb/Kweb.kt

Clean up a couple little things.

d248bb5

sanity suggested changes May 13, 2021

View reviewed changes

Derek52 added 2 commits May 13, 2021 15:42

Remove dead code and make requested changes.

334165d

Merge remote-tracking branch 'upstream/master' into Kotlin_Serializat…

a88efdb

…ion_Migration

sanity marked this pull request as ready for review May 13, 2021 23:06

sanity merged commit ae7382a into kwebio:master May 13, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Kotlin serialization migration #190

Kotlin serialization migration #190

Derek52 commented Apr 10, 2021

sanity commented Apr 11, 2021

Derek52 commented Apr 23, 2021

sanity commented Apr 23, 2021

Derek52 commented Apr 25, 2021

sanity commented Apr 25, 2021

sanity Apr 25, 2021

Derek52 Apr 26, 2021

Derek52 commented Apr 26, 2021

sanity commented Apr 26, 2021

Derek52 commented Apr 26, 2021

Derek52 commented May 7, 2021 •

edited

Loading

sanity left a comment

sanity May 13, 2021

Derek52 May 13, 2021

sanity May 13, 2021

sanity May 13, 2021

sanity May 13, 2021

sanity May 13, 2021

Derek52 May 13, 2021

sanity May 13, 2021

sanity May 13, 2021

		compile 'org.jetbrains.kotlinx:kotlinx-serialization-json:1.1.0-RC'
		//implementation 'org.jetbrains.kotlin:kotlin-reflect:1.5.0'

Kotlin serialization migration #190

Kotlin serialization migration #190

Conversation

Derek52 commented Apr 10, 2021

sanity commented Apr 11, 2021

Derek52 commented Apr 23, 2021

sanity commented Apr 23, 2021

Derek52 commented Apr 25, 2021

sanity commented Apr 25, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Derek52 commented Apr 26, 2021

sanity commented Apr 26, 2021

Derek52 commented Apr 26, 2021

Derek52 commented May 7, 2021 • edited Loading

sanity left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Derek52 commented May 7, 2021 •

edited

Loading