DCDM - more checks needed

[kieranjol]

We have a sequence where one corrupted tif resulted in DCP-O-Matic not being able to make a DCP. After doing an audit in exiftool, we found a corrupt frame. ClairMeta approved this sequence as the filesize, mimetype and sequence continuity were all ok. I suggest some further probing, for example:
Width/Height, Bit Depth , some sort of check that a valid tiff header is present. Whatever is the least intensive for the greatest chance of detecting corruption.
Here's the exiftool report on the corrupt frame:

exiftool E:\DCDM\video\Reel_3\DCDM_Tiff16bit-XYZ_DCI2K-Scope_Reel03_075580.tif
ExifTool Version Number         : 11.19
File Name                       : _DCDM_Tiff16bit-XYZ_DCI2K-Scope_Reel03_075580.tif
Directory                       : E:/DCDM/video/Reel_3
File Size                       : 10 MB
File Modification Date/Time     : 2019:02:08 14:41:08+00:00
File Access Date/Time           : 2019:02:08 14:41:08+00:00
File Creation Date/Time         : 2019:02:08 14:41:08+00:00
File Permissions                : rw-rw-rw-
File Type                       : TIFF
File Type Extension             : tif
MIME Type                       : image/tiff
Exif Byte Order                 : Little-endian (Intel, II)

and here's the previous frame which is fine:

ExifTool Version Number         : 11.19
File Name                       : DCDM_Tiff16bit-XYZ_DCI2K-Scope_Reel03_075579.tif
Directory                       : New folder
File Size                       : 10 MB
File Modification Date/Time     : 2018:05:11 02:23:31+01:00
File Access Date/Time           : 2020:03:10 09:44:41+00:00
File Creation Date/Time         : 2020:03:10 09:44:41+00:00
File Permissions                : rw-rw-rw-
File Type                       : TIFF
File Type Extension             : tif
MIME Type                       : image/tiff
Exif Byte Order                 : Little-endian (Intel, II)
Image Width                     : 2048
Image Height                    : 858
Bits Per Sample                 : 16 16 16
Compression                     : Uncompressed
Photometric Interpretation      : RGB
Strip Offsets                   : 8
Samples Per Pixel               : 3
Rows Per Strip                  : 858
Strip Byte Counts               : 10543104
Planar Configuration            : Chunky
Image Size                      : 2048x858
Megapixels                      : 1.8

Best,

Kieran.

[remia]

Hi @kieranjol ,

Just to be sure, what version of ClairMeta where you using, and was it the DSM or DCDM checker ?

In the latest versions, we no longer use Mime type as it was found to be unreliable, we check for:

• File name coherence
• Increasing frame number with no gap
• File size within a tolerance (to account for different headers size if images comes from different software)
• For DCDM, there is additional metadata checks, relying on mediainfo to probe the files, see https://github.com/Ymagis/ClairMeta/blob/develop/clairmeta/settings.py#L211 for a listing.

That being said we could add more and also make sure there is no change in the way mediainfo extract those that would make ClairMeta outdated. Would be nice to have the corrupted frame and another valid one (no matter which). Will probably look into this next week if there is a need.

Thanks

[kieranjol]

Hi, thanks for getting back so quick and I apologise for having so many things missing from my report.
We were using the DCDM checker.
python -m clairmeta.cli check -type dcdm "C:\Users\blaaa\New folder"

and i'd say i't sht latest version cos we reinstalled lately and pip install -U clairmeta says it's the ;latest version:

C:\Users\blaaa>pip show clairmeta
Name: clairmeta
Version: 1.0.5
Summary: Digital Cinema Package (DCP) probing and checking utility
Home-page: https://github.com/Ymagis/ClairMeta
Author: Ymagis
Author-email: [email protected]
License: BSD
Location: c:\users\blaaa\appdata\roaming\python\python37\site-packages
Requires: lxml, pyopenssl, shutilwhich, dicttoxml, python-dateutil, xmltodict, pycountry, six
Required-by:

I think the mediainfo checks are not running on my end - is there a chance that this is only in git-master but not in the latest pip version?

I'll shoot you an email now too.

Thanks!

[remia]

It should be deployed!

Maybe mediainfo was not found on the host, but you should see warning in that case. Or there is a bug somewhere / version incompatibility, will have to look at the files when you send them and let you know.

[kieranjol]

Thanks remi! Mediainfo is definitely available in.$PATH as its involved in almost all our scripts... K