Psych::SyntaxError on reify

[brodock]

I'm getting a Psych error on reify command with one specific object:

Psych::SyntaxError: (): found unexpected end of stream while scanning a quoted scalar at line 2 column 15

I have no clue about how to proceed to debug it, could you please give some feedback?

[brodock]

I also found that #194 could be a viable alternative, but need advice about how to migrate existing data to a new serializer.

[batter]

Sounds like you may be having an issue where you were using Syck but then upgraded to a newer version of Ruby where Psych is the default parser. There are inconsistencies between Psych and Syck that can sometimes cause errors when one of the parsers attempts to parse a string that was dumped by the other parser.

See #70 for more info about this type of thing, however, I would recommend trying to force Syck as your parser for your application and seeing if that fixes things. An easy way to accomplish this in a Rails application is to put this line in config/application.rb (right underneath the Bundler.require block):

YAML::ENGINE.yamler= 'syck'

As you mentioned, you could use the built in JSON serializer to handle your serialization, however, you would need to write some sort of a script to convert/handle pre-existing data.

[brodock]

I found what was wrong... it wasn't anything with Syck or Psych (as I started with Psych since the beginning... the reason is what I suggest on #248 (the field wasn't big enough to handle the serialized object).

The following sql query help identifing the offenders:

SELECT id, item_type, item_id, LENGTH(object) as object_size FROM versions WHERE LENGTH(object) = 65535;

The suggestion I gave on #248 would solve it for everybody and have no side-effects except for 2 extra bytes per entry, which is essentially irrelevant.

this is the right way to change the field to force it to use LONGTEXT on MySQL:

    # put this inside a migration:
    change_column :versions, :object, :text, limit: 4.gigabytes-1 # MySQL: LONGTEXT

[batter]

Good to know. Sounds like you found the solution so I'm closing this. Seems like it's a database specific problem as not all databases have this restriction on their text fields.

[brodock]

@batter sure, thanks... could you please give me a feedback on #248 if you are going to accept the suggested solution to the problem?

I truly believe that it's a reasonable alternative without any side effect. It also comply to the principle of least surprise.