@mirabilos

Actually, you're right. The native lossless image format in PDF isn't PNG. I'm not totally sure what it is.
pdfimages just says "image," not "PNG", "TIFF", or "PPM"

rld@Intrepid:tmp$ pdfimages -list foo.pdf
page num type width height color comp bpc enc interp object ID x-ppi y-ppi size ratio
--------------------------------------------------------------------------------------------
1 0 image 2040 2040 rgb 3 8 image no 7 0 96 96 3155K 26%
1 1 smask 2040 2040 gray 1 8 image no 7 0 96 96 8188B 0.2%

@mirabilos @rl_dane can't you embed everything? Wasn't there at least one printer manufacturer that embedded firmware updates?

@kabel42

ocrmypdf doesn't even use JBIG by default, so I have no idea how that happened. But that is what happened.

@pixx

no, ocrmypdf just performs the OCR (using tesseract) and inserts it as textual metadata with the original images intact.

Someone suggested it may be using JBIG compression (lossy, cursed) for the image, but that would be weird! I've never seen ocrmypdf compress that well before.

If I had thought of it, I'd have looked to see if the resultant PNG file (once extracted) was the same as the original going in, but I don't think I have the intermediary files anymore.

@natevw

To your first point, you're absolutely right. Thresholding yeilds far more than an 8:1 compression because PNG is far more able to crunch bilevel graphics vs. grayscale.

To your second point, you're describing the #JBIG lossy compressor for scanned documents and monochrome images, and yeah, that's super cursed. I'd be surprised if that's what ocrmypdf is doing, but it's possible? ¯\_(ツ)_/¯

@mctwist

That would be a very strange edge case where expanding the data stream into base64 somehow exposed regularities that the compression algorithm somehow missed in the original data.

I've personally never seen base64/uuencoded files become smaller than the original files when compressed. (compared to the original files compressed the same way)

@Screwtapello

The fax compression algorithms are very limited, designed for a time where RAM was scarce. They basically compressed a couple rows of pixels at a time, nothing more, AFAIK.

There is #JBIG, which is a LOSSY 1-bit monochrome image compression algorithm. Yes, it is exactly as cursed as that sounds, there have been many cases where numbers and figures were changed by JBIG because a 6 looked like an 8. Horrifying. XD

I'm not totally sure what happened in this example, because I realized after I posted this that the pdfimages utility is converting whatever the PDF stored the images as into PNGs, not just extracting any embedded PNGs it finds.