Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parser: allow user-defined rows_event decode func #737

Merged
merged 2 commits into from
Nov 16, 2022

Conversation

GMHDBJD
Copy link
Contributor

@GMHDBJD GMHDBJD commented Nov 16, 2022

add RowsEventDecodeFunc in BinlogSyncerConfig that allow user-defined rows_event decode func

Copy link
Collaborator

@lance6716 lance6716 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also we have planed some use cases of this feature, such as skipping decoding rows events of uninterested table. But in order to do that, we need to know the table name of this RowsEvent. In the original Decode function, the table name is written to RowsEvent.Table by accessing private members

	e.TableID = FixedLengthInt(data[0:e.tableIDSize])
...
	e.Table, ok = e.tables[e.TableID]

and our customized RowsEventDecodeFunc can not do that.

Maybe we should adjust more code? Or we inject the decoding function at e.decodeRows?

@@ -40,6 +40,8 @@ type BinlogParser struct {
useDecimal bool
ignoreJSONDecodeErr bool
verifyChecksum bool

rowsEventDecodeFunc func(*RowsEvent, []byte) error
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we can set the default value to (*RowsEvent).Decode, so in parseEvent we don't need to check it many times.

@GMHDBJD
Copy link
Contributor Author

GMHDBJD commented Nov 16, 2022

and our customized RowsEventDecodeFunc can not do that.

rowsEventDecodeFunc = func(re *replication.RowsEvent, data []byte) error {
			_, err := re.DecodeHeader(data)
			tb := &filter.Table{
				Schema: string(re.Table.Schema),
				Name:   string(re.Table.Table),
			}
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants