Oodle Texture "preserve extremes" rules

gistfile1.txt

{
	BC7Flags flags = in_flags;

	// Preserve extremes mode.
	//
	// This mode preserves values of 0 and 255 in the alpha channel exactly. In general,
	// we can do this in any one channel, but restricting this to alpha makes the interface
	// simpler, is consistent with what we do for BC3, and doesn't seem like a signfiicant
	// limitation for the user.
	//
	// We keep track of whether we have pixels that have either A=0 or A=255 at all. If not,
	// the constraint is not active and we can encode the block regularly; we just end up
	// turning off "preserve extremes" in that case.
	//
	// To be more specific, we break down "preserve extremes" into "preserve A=0 pixels" and
	// "preserve A=255 pixels" flags internally, and clear the flag bit for either constraint
	// when no pixels in the block match that description.
	//
	// Next, let's consider our options for the various block modes in case we have a block
	// with active constraints:
	//
	// - Modes 0-3 decode with A=255 for every pixel. These modes may be chosen as long as there
	//   are no pixels that have A=0, although in practice we only use them when every source
	//   pixel has A=255 anyway.
	// - Modes 4 and 5 code three channels as a group (the "vector channels" in this code), and a
	//   fourth channel (the "scalar channel") separately. It can be selected per block which
	//   channel is the scalar channel. For blocks that need to preserve extremes, we force the
	//   scalar channel to be A, which gives us the necessary freedom to preserve it exactly.
	// - Modes 6 and 7 can have a non-trivial A channel, but the channels are not separate;
	//   all four are interpolated as a group. To keep things simple, we only support them in
	//   an important special case: mode 6 and 7 may be used for blocks with constant A value
	//   for all pixels when that A value is either 0 or 255. This results in a few restrictions
	//   on the endpoints (mainly, it forces our choice of pbits) but does not result in individual
	//   per-pixel constraints on the indices and is fairly straightforward.
	//
	//   It would also be possible to use mode 6 and 7 more generally by including endpoint values
	//   with A=0 or A=255, and then forcing those pixels to use the index values that give the
	//   correct result (no matter what the RGB values are), but this is both more complicated
	//   and likely to result in high RGB errors (thus unlikely to get chosen), so we currently
	//   don't bother with it.
	//
	// We could similarly allow mode 4 and 5 blocks with constant A to use another channel as the
	// scalar channel, since we can reproduce a constant all-0 or all-255 channel easily in the
	// vector channels too, but we're currently not using this.
	//
	// In short, our bread and butter for "preserve extremes" are modes 4 and 5, which are like
	// BC3 or BC5 in that we have fully independent channels with independent interpolation weights.
	// It turns out that in these modes, all we really need to do to ensure pixels with A=0 and A=255
	// can be preserved exactly is to make A=0 or A=255 respectively be included in the endpoint range.
	// That is, a block containing A=255 pixels should have its higher A endpoint be 255, and a block
	// containing A=0 pixels shoulld have its lower A endpoint be 0. (For a block containing both, this
	// fully determines the A endpoint values).
	//
	// There is not much for us to do beyond only allowing certain modes and endpoint choices
	// when A={0,255} pixels are present. As such, almost all of the "preserve extremes" logic happens
	// either here or in calc_endpoints_and_pbits above.
	if ( flags & BC7ENC_PRESERVE_EXTREMES )
	{
		const BC7PartitionInfo& nfo = in_prep->info_1sub; // 1-subset modes

		// If the block bbox doesn't have a=0 as its lower bound, there's no a=0 to
		// preserve.
		if ( nfo.bbox[0][0].a != 0)
			flags &= ~BC7ENC_PRESERVE_A0;

		// Likewise, we can turn off A=255 preservation if we don't actually see that value.
		if ( nfo.bbox[0][1].a != 255 )
			flags &= ~BC7ENC_PRESERVE_A255;

		// Do we have anything to preserve in this block after this check?
		if ( flags & BC7ENC_PRESERVE_EXTREMES )
		{
			// We have either a low A of 0 or a high A of 255.

			// We already don't allow modes 0-3 (which are always opaque) for blocks
			// that contain non-255 alpha, but be explicit here:
			if ( nfo.bbox[0][0].a != 255 ) // lower bound not also 255
				flags |= BC7ENC_DISABLE_MODE0123;

			// Modes 4/5 send alpha fully separate and don't have p-bits, so they're
			// viable candidates for sure, we just need to make sure the alpha endpoints
			// are what we need them to be, and we need to make sure the channel rotate
			// leaves alpha as the scalar channel.

			// If all alpha values are same (either all 0 or all 255), we can use modes 6
			// and 7, with the caveat that we need to set up the p-bits to make sure
			// we hit those values exactly.
			//
			// If the endpoints aren't the same, we can't generally use these modes without
			// much more complicated types of constraints that we don't want to deal with
			// right now.
			if ( nfo.bbox[0][0].a != nfo.bbox[0][1].a )
				flags |= BC7ENC_DISABLE_MODE67;
		}
	}

	return flags;
}