Instead process the remaining as a slightly oversized chunk. But if the remaining bytes are less than, say, N=50k, then don't tail off the M bytes.
Compute the md5sums of each chunk and store them in a master list called the goldlist.Refer to the first tailing (that contains the end of the file) as the end-chunk. Chunk up the file starting at their end by tailing off, say, M=100k bytes.
Let's refer to the current backup drive as the gold drive. It assumes that files end with the image itself, that all the metadata is at the beginning of the file. Given concerns about image quality it is wise to run this only once.Ī possible solution that just came to mind. You need to build the checksums for every image every time. If you store the checksums and then re-run the -strip after an update of zlib or libjpeg you may end with completely different checksums. You just need to cross fingers that the more exotic image formats can be created with a single implementation (or have a rigid file format).ĭisclaimer: This will work to compare the checksums between themselves. exec mogrify -strip -taint -compress JPEG \ Īnd then the checksumming would be the same. type f -name '*.jp*g' -o -type f -name '*.JP*G' \ This is because different library implementations will produce different compressed results, ImageMagick can perform a compression unification.Ĭommon types are easy because the OS has libraries to read and write them. I would go with ImageMagick for most images. My plan is to identify uniqueness based on md5sums after stripping any and all metadata.
Įven though I don't expect to find any corruption/data-rot between different copies of otherwise identical images, I'd like to detect that, as well as differences due to resizing and color changes.
I'm only interested in uniqueness of the image itself and not uniqueness due to differences in, say, the values of exif tags, the presence/absence of a given exif tag itself, embedded thumbnails, etc. Formats include jpg, png, tif, etc., as well as various raw formats (different camera models and manufacturers). I'd like to search my old hard drives for photos that are not on my current backup drive.
0 kommentar(er)
