If we need save objects to file, then recreate the objects from file, or transfer the objects between different systems, we need convert the objects to a stream of bytes, then it can be saved or transfered.
The process that convert an object into data is called serialization, and recreate the object from data is called deserialization.
The serialization system won't know every data formats, so it only convert objects into tables, or recreate objects from lua tables.
Then a data format provider will be used to convert the lua table into the target data format, or load the data and create the lua tables.
The PLoop has provided three data format providers, we'll learn how to use the system through them.
The target format is normal Lua values, so the objects will be converted to the tables without metatable. Actually, the serialization's process is like:
Object -> Lua table -> target format
So the LuaFormatProvider doesn't do anything but just return the middle value directly.
require "PLoop" (function(_ENV)
import "System.Serialization"
__Serializable__()
class "Person" (function(_ENV)
property "Name" { type = String }
property "Age" { type = Number }
end)
ann = Person{ Name = "Ann", Age = 21 }
data = Serialize(LuaFormatProvider(), ann)
print(getmetatable(data)) -- nil
-- __PLoop_Serial_ObjectType Person
-- Age 21
-- Name Ann
for k, v in pairs(data) do
print(k, v)
end
ann = Deserialize(LuaFormatProvider(), data)
-- Person Ann 21
print(getmetatable(ann), ann.Name, ann.Age)
data.__PLoop_Serial_ObjectType = nil
ann = Deserialize(LuaFormatProvider(), data)
-- nil Ann 21
print(getmetatable(ann), ann.Name, ann.Age)
ann = Deserialize(LuaFormatProvider(), data, Person)
-- Person Ann 21
print(getmetatable(ann), ann.Name, ann.Age)
end)The Serialize and Deserialize are static method of the System.Serialization, since we import the class, we can used those methods directly.
In the data, there is a special field named __PLoop_Serial_ObjectType, the object type will be saved in that field, if we remove it, we must provide the class type to the Deserialize method so the system can know which type you want convert the data into.
If you don't need the __PLoop_Serial_ObjectType field, you can change the format provider's property like:
require "PLoop" (function(_ENV)
import "System.Serialization"
__Serializable__()
class "Person" (function(_ENV)
property "Name" { type = String }
property "Age" { type = Number }
end)
ann = Person{ Name = "Ann", Age = 21 }
data = Serialize(LuaFormatProvider{ ObjectTypeIgnored = true }, ann)
print(getmetatable(data)) -- nil
-- Age 21
-- Name Ann
for k, v in pairs(data) do
print(k, v)
end
end)You can also change the field name to other string like
require "PLoop" (function(_ENV)
import "System.Serialization"
Serialization.ObjectTypeField = "PLOOP_TYPE"
__Serializable__()
class "Person" (function(_ENV)
property "Name" { type = String }
property "Age" { type = Number }
end)
ann = Person{ Name = "Ann", Age = 21 }
data = Serialize(LuaFormatProvider(), ann)
-- PLOOP_TYPE Person
-- Age 21
-- Name Ann
for k, v in pairs(data) do
print(k, v)
end
end)But you must keep using the same name for the serialization and deserialization.
The Lua table is only the middle value of the serialization process, it can be used in several scenario, but in mostly condition, we convert the object into string, so it can be saved to anyplaces.
require "PLoop" (function(_ENV)
import "System.Serialization"
__Serializable__()
class "Person" (function(_ENV)
property "Name" { type = String }
property "Age" { type = Number }
end)
ann = Person{ Name = "Ann", Age = 21 }
data = Serialize(StringFormatProvider{ ObjectTypeIgnored = true }, ann)
-- {Age=21,Name="Ann"}
print(data)
ann = Deserialize(StringFormatProvider(), data, Person)
-- Person Ann 21
print(getmetatable(ann), ann.Name, ann.Age)
end)We also can get a good format result by change the format provider's property:
require "PLoop" (function(_ENV)
import "System.Serialization"
__Serializable__()
class "Person" (function(_ENV)
property "Name" { type = String }
property "Age" { type = Number }
end)
ann = Person{ Name = "Ann", Age = 21 }
data = Serialize(StringFormatProvider{ ObjectTypeIgnored = true, Indent = true }, ann)
-- {
-- Age = 21,
-- Name = "Ann"
-- }
print(data)
end)There is two more property to change the link break char or indent char:
- LineBreak - the line break, default '\n'
- IndentChar - the indent char, default '\t'
The most widely used data format through the internet.
require "PLoop" (function(_ENV)
import "System.Serialization"
json = [==[
{
"debug": "on\toff",
"nums" : [1,7,89,4,5],
"char" : { "a": 11, "b": 12 },
}]==]
-- deserialize json data to lua table
data = Deserialize(JsonFormatProvider(), json)
-- {
-- nums = {
-- [1] = 1,
-- [2] = 7,
-- [3] = 89,
-- [4] = 4,
-- [5] = 5
-- },
-- debug = "on off",
-- char = {
-- a = 11,
-- b = 12
-- }
-- }
print(Serialize(StringFormatProvider{ Indent = true }, data))
end)There is a special rule for the JSON data format, if it check the object type is array struct, it won't need to check the elements to find this is an array, and it the type is member or dictionary struct, there is also no need to check.
Not all the PLoop type datas are serializable, here is the rules:
-
The enum types are always serializable.
-
The class types are serializable only when its marked with
System.Serialization.__Serializable__attribute. If the class type is generated from a template, the class type could be serializable if the template is serializable and all template parameters are serializable when they are types. -
The custom struct types are serializable only when it has base struct type and the base struct type is serializable or the struct type is marked with
System.Serialization.__Serializable__attribute. If the struct type is generated from a template, the struct type would be serializable if the template is serializable and all template parameters are serializable when they are types. -
The array struct types are serializable only when its element type is serializable.
-
The member struct types are serializable only when all its member types are serializable(exclude the members marked with
System.Serialization.__NonSerialized__attribute). -
The dictionary struct types are serializable only when its key type is a serializable custom struct type and its value type is also serializable.
We can use System.Serialization.__NonSerialized__ attribute to mark class property or struct member as non-serialized data.
The System has provide several serializable custom struct types:
- System.Boolean
- System.String
- System.Number
- System.AnyBool
- System.NEString
- System.PositiveNumber
- System.NegativeNumber
- System.Integer
- System.NaturalNumber
- System.NegativeInteger
- System.Guid
require "PLoop"
require "PLoop.System.Web"
PLoop(function(_ENV)
import "System.Serialization"
import "System.Web"
struct "Person" (function(_ENV)
name = String
age = Number
Children= struct { Person }
end)
per = Person{
name = "King", age = 32,
Children= {
Person("Ann", 9),
Person("Ben", 6),
}
}
-- {"name":"King","age":32,"Children":[{"name":"Ann","age":9},{"name":"Ben","age":6}]}
print(Serialize(JsonFormatProvider{Indent = false}, per, Person))
__Serializable__()
class "Student" (function(_ENV)
property "Name" { type = String }
property "Class"{ type = String }
end)
ben = Student{ Name = "Ben", Class = "A-5" }
ann = Student{ Name = "Ann", Class = "A-6" }
-- [{"Class":"A-5","Name":"Ben"},{"Class":"A-6","Name":"Ann"}]
print(Serialize(JsonFormatProvider{ Indent = false}, { ben, ann }))
end)For struct types, normally only custom struct need the __Serializable__ attribute. Since the struct type value has no meta-table, the system can't figure out the type, so we need pass the type to the Serialize and Deserialize.
For simple class like the Student, we also only need mark it with the __Serializable__ attribute, all property without __NonSerialized__ attribute will be saved to the data.
Not all the serializable class are so simple, in some condition, the system can't figure out the true data that need to be serialized(The system can only handle the class's properties not super class's), or the classes have constructor defined(not support the init-table), the system won't know how to create the object.
So, the type should provide the information by itself.
A class need to do custom serialization must extend the System.Serialization.ISerializable interface, the interface defined a method Serialize, it receive one argument System.Serialization.SerializationInfo, the class have two methods:
- obj:SetValue(name, value, valueType) -- Store the data to the SerializationInfo.
- obj:GetValue(name, valueType) -- Get the data from the SerializationInfo with data type.
So the class object can save the data to the SerializationInfo object, and then the system can use the SerializationInfo for serialization.
The class also should have an overloaded constructor that only accept the SerializationInfo as argument. Here is an example :
require "PLoop"
PLoop(function(_ENV)
import "System.Serialization"
namespace "Test"
__Serializable__()
class "Person" (function (_ENV)
extend "ISerializable"
property "Name" { Type = String }
property "Age" { Type = Number }
function Serialize(self, info)
-- Set the value with type
info:SetValue("name", self.Name, String)
info:SetValue("age", self.Age, Number)
end
__Arguments__{ String, Number }
function Person(self, name, age)
self.Name = name
self.Age = age
end
__Arguments__{ SerializationInfo }
function Person(self, info)
-- Get the value with type
this(self, info:GetValue("name", String) or "Noname", info:GetValue("age", Number) or 0)
end
end)
__Serializable__()
class "Student"(function (_ENV)
inherit "Person"
property "Score" { Type = Number }
function Serialize(self, info)
-- Set the child class's value with type
info:SetValue("score", self.Score, Number)
-- Call the super's Serialize
super.Serialize(self, info)
end
__Arguments__{ SerializationInfo }
function Student(self, info)
super(self, info)
-- Get child class value with type
self.Score = info:GetValue("score", Number)
end
__Arguments__.Rest()
function Student(self, ...)
super(self, ...)
end
end)
Ann = Student("Ann", 16)
Ann.Score = 81
data = Serialization.Serialize( StringFormatProvider(), Ann)
-- {__PLoop_Serial_ObjectType="Test.Student",name="Ann",age=16,score=81}
print(data)
p = Serialization.Deserialize( StringFormatProvider(), data)
-- Test.Student Ann 16 81
print( getmetatable(p), p.Name, p.Age, p.Score)
end)The Serialization system only provide the object <-> table conversion, so the table <-> data must be done by a data format provider.
A data format provider must inherit the System.Collections.FormatProvider and provide two method, Here is an example of a FormatProvider:
require "PLoop"
require "PLoop.System.IO"
PLoop(function(_ENV)
import "System.Serialization"
import "System.IO"
-- Serialize simple objects that only contains one-level key-value pairs
class "SimpleFormatProvider" (function(_ENV)
inherit "FormatProvider"
--- Serialize the common lua data to the target format and return the target data
__Arguments__{ Any }
function Serialize(self, data)
local result = {}
for k, v in pairs(data) do
if k ~= Serialization.ObjectTypeField then
if type(v) == "string" then
table.insert(result, ("%s=%q"):format(k, v))
else
table.insert(result, ("%s=%s"):format(k, tostring(v)))
end
end
end
return "{" .. table.concat(result, ",") .. "}"
end
--- Serialize the common lua data to the target format and sent them with the write
__Arguments__{ Any, Function }
function Serialize(self, data, write)
write("{")
local first = true
for k, v in pairs(data) do
if k ~= Serialization.ObjectTypeField then
if first then
first = false
else
write(",")
end
if type(v) == "string" then
write(("%s=%q"):format(k, v))
else
write(("%s=%s"):format(k, tostring(v)))
end
end
end
write("}")
end
--- Serialize the common lua data to the target format and sent them with the writer
__Arguments__{ Any, System.Text.TextWriter }
function Serialize(self, data, writer)
writer:Write("{")
local first = true
for k, v in pairs(data) do
if k ~= Serialization.ObjectTypeField then
if first then
first = false
else
writer:Write(",")
end
if type(v) == "string" then
writer:Write(("%s=%q"):format(k, v))
else
writer:Write(("%s=%s"):format(k, tostring(v)))
end
end
end
writer:Write("}")
end
__Arguments__{ Function }
function Deserialize(self, read)
return loadstring("return " .. List(read):Join())()
end
--- Deserialize the data from the reader to common lua data
__Arguments__{ System.Text.TextReader }
function Deserialize(self, reader)
return loadstring("return " .. reader:ReadToEnd())()
end
--- Deserialize the data to common lua data
__Arguments__{ Any }
function Deserialize(self, data)
return loadstring("return " .. data)()
end
end)
__Serializable__()
class "Student" (function(_ENV)
property "Name" { type = String }
property "Class"{ type = String }
end)
ben = Student{ Name = "Ben", Class = "A-5" }
-------------------------
-- Direct access
data = Serialize(SimpleFormatProvider(), ben)
-- {Class="A-5",Name="Ben"}
print(data)
ben = Deserialize(SimpleFormatProvider(), data, Student)
-- Student Ben A-5
print(Class.GetObjectClass(ben), ben.Name, ben.Class)
-------------------------
-- Direct accessTextReader & TextWriter
strWriter = System.Text.StringWriter()
with(strWriter)(function(writer)
Serialize(SimpleFormatProvider(), ben, writer)
end)
data = strWriter.Result
print(data)
strReader = System.Text.StringReader(data)
ben = Deserialize(SimpleFormatProvider(), strReader, Student)
-- Student Ben A-5
print(Class.GetObjectClass(ben), ben.Name, ben.Class)
-------------------------
-- Function As Iterator
__Iterator__() function getStream(obj)
Serialize(SimpleFormatProvider(), obj, coroutine.yield)
end
-- {
-- Class="A-5"
-- ,
-- Name="Ben"
-- }
for k in getStream(ben) do print(k) end
ben = Deserialize(SimpleFormatProvider(), getStream(ben), Student)
-- Student Ben A-5
print(Class.GetObjectClass(ben), ben.Name, ben.Class)
end)For the Serialize, there are three overloads:
- Any -- Only serialize the object or value, and return the result.
- Any, Function -- Serialize the object or value and send all the middle results to the function like a stream, no value should be returned.
- Any, TextWriter -- Serialize the object or value and send all the middle results to the text writer, , no value should be returned. Here we use the StringWriter to build a string as the result.
For the Deserialize, there are three overloads:
- Any -- Only deserialize the data and return the value.
- Function -- Get the data from the iterator and deserialize the data, then return the value. The function must be an iterator, best generated by
__Iterator__attribute. - TextReader -- Get the data from the text reader and deserialize the data, then return the value.
We may learn more about the TextWriter and TextReader in System.IO part.