What were they thinking? They basically encoded (I think) the sum of the two images as the first image, plus one of the two images as the second (that's your one-stop difference). The result is that the auxiliary image takes a lot more space than it actually needs to. It seems to me that they should have combined sum-difference encoding, sign-magnitude encoding, and run-length encoding, as follows:

Sum-difference encoding.

• Encode one full image as the sum of the left + right images. This is standard RAW data.
• Encode a second image as the difference between the left + right images.

What's important to note, at this point, is that except for the outliers where a pixel's values are very different on both sides (massively out-of-focus areas?), most of your values here should be very small positive or negative numbers, with many pixel values being nearly all zeroes or ones.

It is also important to understand that, assuming you don't have too few bits to work with, this translation is lossless. You can add the resulting parts together and get L+R+L-R = 2L. Shift by one bit, and you have L. Similarly, you can subtract these from each other and L+R-(L-R) = L+R-L+R = 2R. Shift by one bit, and you have R.

Sign-magnitude encoding:

• For each negative value, compute the absolute value (or just invert all of the bits) and reset the sign bit to its original value. This ensures that small negative numbers are also almost all zeroes.
• Optional for lossless JPEG: Rotate each value one bit to the left so that the sign bit is on the right end. This ensures that the left end of each number is almost all zeroes.

By making the leftmost digits be zero, various compression algorithms work better, including the lossless JPEG that they're already using, I think. But the real benefit of having mostly zeroes comes when you do bitwise run-length encoding. (And the one-bit rotation is irrelevant if you do this.)

Bitwise run-length encoding

• Write the topmost bit of every pixel to a file using run-length encoding. This will compress to a very small number of bytes because it will be all zeroes, so the data basically just says "30 million zero bits".
• Repeat for the next bit. "10,000 zero bits, one 1 bit, 25000 zero bits, two 1 bits...".
• Repeat until done.

I suspect that this relatively small amount of simple bit math would have been enough to reduce the dual-pixel RAW penalty enough to make it practical for everyone to just turn on dual-pixel RAW and leave it on. The CPU impact should be minimal—even my quick, hackish, naïve, non-vectorized implementation of most of this math ran 50 million times in under a second on a modern CPU, so the big processing hit should be tiny, too.

Am I missing something subtle? Are the differences much larger than I think they are? Or did nobody at Canon take Dr. Langdon's data compression course (RIP)? :-)

Pedantically, yes, it can decode the fractional image. But that's really not the interesting part of supporting dual-pixel RAW. After all, that secondary image is only the fractional image from one half of each subpixel, which isn't something you would ever want to use by itself as an image. For example, if you look at sufficiently out-of-focus areas, IIRC, you'll see a semicircle where the bokeh ball should be. It looks ridiculous by itself.

The half image is only interesting in the context of software specifically designed to take that data and combine it with the full image in interesting ways. The code required to decode the image is at most a tiny fraction of the code required to support dual-pixel RAW in any *meaningful* way.

Encoding the primary (summed) image in the same way that they've always encoded it makes perfect sense from a compatibility perspective, because that enables RAW decoders to add basic support for the photos with only minimal effort. Encoding the fractional image in the same way? That's just wasting space, and that really adds up. As a result, most users will miss the opportunity to take advantage of this potentially powerful feature. And that's a shame—particularly given that any compatibility issue would be completely moot as long as Canon provided a reference implementation under a permissive license....

I was wondering...
If final image is from 2 sub images slightly shifted in focus, does that mean that the final A+B image will have a deeper depth of field than an image from a similar camera without dual pixel raw (even if the DPR recording is disabled or for out-of-camera jpeg) ?
Does it also mean that by changing the mix ratio between A and B at sum time we could change the depth of field afterwards?

Raoul