Deserialize JSON Exception?

[yuessir]

Hi all, this is my code , one property of the class,
public Dictionary<string, object> InfoArgs { get; set; }

First time,I use the method SaveStateAsync ,and the .JSON file in the MS blob,
and then I modified the .JSON file and replaced the original JSON file.
Next time ,I start the VS debug mode ,set as project with SILO host,and got the excption "System.Runtime.Serialization.SerializationException: Type 'Newtonsoft.Json.Linq.JObject' in Assembly 'Newtonsoft.Json, Version=7.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed' is not marked as serializable."

I found similar issue in the https://gitter.im/dotnet/orleans/archives/2015/07/21
Our MS orleans version is V1.2.3, Newtonsoft is V7+
I am not sure that is my code has some problem or others.
Anybody have some ideas?Thanks.

[sergeybykov]

Have you found the discussion in #299?

[yuessir]

@sergeybykov Thanks for providing the discussion :) . I got some useful information. But I have a question , why the Orleans can get the actor successfully at the first time?then I replaced the original JSON file with the modified then threw an exception?

[sergeybykov]

Doesn't Newtonsoft throw the same exception if you try to deserialize the modified JSON even outside of Orleans?

[yuessir]

@sergeybykov
Deserializing the modified JSON outside of Orleans is okay , I just modified some value.
Original JSON file in the blob : getting the actor without any problem( deserializing process seems ok)
Modified JSON file in the blob : throw the exception..

this.State.InfoArgs =command.InfoArgs .ToJson();//Dictionary InfoArgs property to JsonConvert.SerializeObject

[centur]

Which storage provider are you using, if any ? Also can you describe or provide an example of how you modified JSON ?

Take a note that the serialization configuration which is used by Orleans is very strict. I think it's like this (it might be from older version of orleans, but key point here is TypeNameHandling = TypeNameHandling.All

var serializerSettings = new JsonSerializerSettings
    {
        TypeNameHandling = TypeNameHandling.All,
        DateFormatHandling = DateFormatHandling.IsoDateFormat,
        DefaultValueHandling = DefaultValueHandling.Ignore,
        MissingMemberHandling = MissingMemberHandling.Ignore,
        NullValueHandling = NullValueHandling.Ignore,
        ConstructorHandling = ConstructorHandling.AllowNonPublicDefaultConstructor,
        Formatting = Newtonsoft.Json.Formatting.None
    };

Try to deserialize your modified json using these settings and let us know about the results.
Small hint - with these settings above your json would look like this - with types full names
:

{
  "$id": "1",
  "$type": "DBCloud.ActorInterfaces.ExternalJobs.ExternalJobsQueueState, DBCloud.ActorInterfaces",
  "PendingJobs": {
    "$id": "2",
    "$type": "System.Collections.Generic.Dictionary`2[[System.String, mscorlib],[DBCloud.ActorInterfaces.ExternalJobs.ExternalJobRecord, DBCloud.ActorInterfaces]], mscorlib"
  },
  "ProcessingJobs": {
    "$id": "3",
    "$type": "System.Collections.Generic.Dictionary`2[[System.String, mscorlib],[DBCloud.ActorInterfaces.ExternalJobs.ExternalJobRecord, DBCloud.ActorInterfaces]], mscorlib"
  }
}

we found that messing with it causes Json.NET go pretty nuts

[yuessir]

@centur
The Azure blob as the storage provider,
I tried to build another application to simulate the case ,

    public async Task SetCouponInfo(ConponInfoTest c)
        {
            this.State.Amount = c.Amount;
            this.State.CouponId = c.CouponId;
            this.State.Name = c.Name;
            this.State.Args = TocCouponDictionary();

            await this.SaveStateAsync();
        }
 public Task<ConponInfoTest> GetCouponInfo()
        {
            // Dictionary<string, object> newArgs = new Dictionary<string, object> { { "test1", this.State.Args["test1"] }, { "test2", this.State.Args["test2"] } };

            ConponInfoTest ct = new ConponInfoTest
            {
                Amount = this.State.Amount,
                Args = this.State.Args,
                CouponId = this.State.CouponId,
                Name = this.State.Name
            };
            return Task.FromResult(ct);
        }

        private static Dictionary<string, object> TocCouponDictionary()
        {
              var t1 = new test1 { ErrorId = "155", ErrorMsg = "erroetest" };
            var t2 = new test2 { StatusCode = "200", StatusDesc = "ok" };
            //var tt1 = JsonConvert.DeserializeObject<test1>(t1.ToJson());
            new JavaScriptSerializer().Serialize(t1);
            new JavaScriptSerializer().Serialize(t2);
            //var tt2 = JsonConvert.DeserializeObject<test2>(t2.ToJson());
            return new Dictionary<string, object>
            {
                { "test1", t1 },
                { "test2", t2 }
            };
        }

   [Serializable]
    public class test1
    {
        public string ErrorId { get; set; }
        public string ErrorMsg { get; set; }
    }
    [Serializable]
    public class test2
    {
        public string StatusCode { get; set; }
        public string StatusDesc { get; set; }
    }

original JSON file in the MS Azure Blob content
{"Data":"{"t1":null,"t2":null,"Amount":"546","Args":{"test1":{"ErrorId":"155","ErrorMsg":"erroetest"},"test2":{"StatusCode":"200","StatusDesc":"ok"}},"CouponId":"AD170340092841DFBFAA56F130A8FD92","Name":"suirr","ETag":null,"LastModified":"2016-09-02T09:31:06.7202125Z","State":null}","Id":"AD170340092841DFBFAA56F130A8FD92","Key":"GrainReference=41df0928ad17034092fda830f156aabf03ffffffadea3364","LongId":0,"TimeStamp":636084054667272295,"Type":"Yu.Domain.Coupon"}

and I deleted original one, and replaced the following :
{"Data":"{"t1":null,"t2":null,"Amount":"546","Args":{"test1":{"ErrorId":"000","ErrorMsg":"erroetest"},"test2":{"StatusCode":"200","StatusDesc":"ok"}},"CouponId":"AD170340092841DFBFAA56F130A8FD92","Name":"suirr","ETag":null,"LastModified":"2016-09-02T09:31:06.7202125Z","State":null}","Id":"AD170340092841DFBFAA56F130A8FD92","Key":"GrainReference=41df0928ad17034092fda830f156aabf03ffffffadea3364","LongId":0,"TimeStamp":636084054667272295,"Type":"Yu.Domain.Coupon"}

Just changed the value "ErrorId:155" to "ErrorId:000", and restart the SILO then threw the exception

"No copier found for object of type Newtonsoft.Json.Linq.JObject. Perhaps you need to mark it [Serializable] or define a custom serializer for it?"

First can parsing json,second dead...:( ,may you explain the situation? Thanks..

[centur]

Sorry for a delayed reply. This doesn't look right. Azure Blob Storage provider, as far as I know, must serialize data with types , are you 100% sure you haven't disabled TypeNameHandling = TypeNameHandling.All in Orleans serialization configuration ? cc @richorama

Also this error with no copier found. It seems that it's related to a bit different problem actually - we had this error when we tried to pass JObject to a grain or read JObject from a grain. It works ok when you're running just a single silo (e.g. in dev or local debugging) but fails when you're running in a cluster - when jObject is being passed to other server - runtime is trying to serialize it ( when it's on a single silo - some optimisations come in play and they conseal this problem )
Try to add this code to your project.

/// <summary> Provides support for serializing JSON values. </summary>
    [RegisterSerializer]
    public class OrleansJsonSerialization
    {
        private static readonly Type[] JsonTypes;

        /// <summary> Initializes static members of the <see cref="OrleansJsonSerialization" /> class. </summary>
        static OrleansJsonSerialization()
        {
            JsonTypes = new[]
            {
                typeof(JObject),
                typeof(JArray),
                typeof(JToken),
                typeof(JValue),
                typeof(JProperty),
                typeof(JConstructor)
            };

            Register();
        }

        /// <summary> The deep copier. </summary>
        /// <param name="original"> The original. </param>
        /// <returns> The copy. </returns>
        public static object DeepCopier(object original) { return original; }

        /// <summary> Serializes an object to a stream. </summary>
        /// <param name="obj"> The object being serialized. </param>
        /// <param name="stream"> The stream to serialize to. </param>
        /// <param name="expected"> The expected type. </param>
        public static void Serialize(object obj, BinaryTokenStreamWriter stream, Type expected)
        {
            var str = JsonConvert.SerializeObject(obj, expected, SerializationSettings.JsonConfig);
            SerializationManager.SerializeInner(str, stream, typeof(string));
        }

        /// <summary> Deserializes a JSON object. </summary>
        /// <param name="expected"> The expected type. </param>
        /// <param name="stream"> The stream. </param>
        /// <returns> The deserialized object. </returns>
        public static object Deserialize(Type expected, BinaryTokenStreamReader stream)
        {
            var str = (string) SerializationManager.DeserializeInner(typeof(string), stream);
            return JsonConvert.DeserializeObject(str, expected, SerializationSettings.JsonConfig);
        }

        /// <summary> Registers this class with the <see cref="SerializationManager" />. </summary>
        public static void Register()
        {
            foreach (var type in JsonTypes)
            {
                SerializationManager.Register(type, DeepCopier, Serialize, Deserialize);
            }
        }
    }

If this helps - the issue is not in the Blob Storage provider...

[yuessir]

@centur
Thanks, I applied the setting you suggested in the class BlobStorageProvider.

CASE1

public class BlobStorageProvider : IStorageProvider
{
        private JsonSerializerSettings settings = new JsonSerializerSettings
        {
            TypeNameHandling = TypeNameHandling.All,
            DateFormatHandling = DateFormatHandling.IsoDateFormat,
            DefaultValueHandling = DefaultValueHandling.Ignore,
            MissingMemberHandling = MissingMemberHandling.Ignore,
            NullValueHandling = NullValueHandling.Ignore,
            ConstructorHandling = ConstructorHandling.AllowNonPublicDefaultConstructor,
            Formatting = Formatting.None
        };
         .......
}

Result:
First time( start the SILO( VS debug mode))
setting the data to the Grain : ok
getting the data from the Grain : ok
and restart the SILO( VS debug mode)
Second time(after modified the json file) :
setting the data to the Grain : ok
getting the data from the Grain : ok
I think the issue is that I used the incorrect JsonSerializerSettings setting.

CASE2

public class BlobStorageProvider : IStorageProvider
{
 private JsonSerializerSettings settings;//all setting is default
......
}

Result:
First time( start the SILO( VS debug mode))
setting the data to the Grain : ok
getting the data from the Grain : ok //why?
and restart the SILO( VS debug mode)
Second time(after modified the json file) :
setting the data to the Grain : ok
getting the data from the Grain : exception

[centur]

Are you implementing your own blob storage provider ?

[centur]

also re: observed behaviour - are you changing the state in the second case when the grain is still in memory ? This part :

Result:
First time
setting the data to the Grain : ok
getting the data from the Grain : ok //why?

When grain is still in memory - normally it doesn't re-read the state, thus you are actually reading the state object that is in memory and was set by your previous Set step - so no re-serialization on read is involved.

Second time - the json you pasted before is not really a json that built-in blob storage provider stores - built-in one stores it with $type information, so it can re-construct exact types that were saved, because, due to Storage interface limitation - Storage provider must make an assignment of Object type to IGrainState.State so your json must contain all the details about the actual types that were uses in serializing original object. You can't have your own arbitrary untyped JSON there ,as Storage Provider doesn't have access to the original state's type information - it must be embedded into stored data itslef