Fixes issues when downmixing to mono or when encoding below fullband