[BUG] Missing spec for delta_log/00000000000000000000.crc files

[felipepessoto]

Bug

Describe the problem

The PROTOCOL doesn't describe the CRC files in delta_log, is it intentional?

Example: 00000000000000000000.crc

{"tableSizeBytes":1234567,"numFiles":123,"numMetadata":1,"numProtocol":1,"protocol":{"minReaderVersion":1,"minWriterVersion":2},"metadata....

Willingness to contribute

The Delta Lake Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the Delta Lake code base?

• Yes. I can contribute a fix for this bug independently.
• Yes. I would be willing to contribute a fix for this bug with guidance from the Delta Lake community.
• No. I cannot contribute a bug fix at this time.

[scottsand-db]

Thanks for making this issue! I believe that @vkorukanti will follow up!

[felipepessoto]

Hi @vkorukanti, do you have any comments on this?

[vkorukanti]

@felipepessoto Where did you see this crc file? cc. @prakharjain09.

[felipepessoto]

Currently it is created by Databricks only.

[felipepessoto]

@prakharjain09 any insights?

[prakharjain09]

Thanks for bringing this up @felipepessoto . We are working on adding support for checksum file in the Delta spec and implementation.