Android TalkBack inconsistency in TextInputLayout

Question

I'm attempting to work around a TalkBack accessibility event issue with a TextInputLayout and TextInputEditText, and I've noticed what gets read, greatly depends on different unrelated factors.

tl;dr (for the lazy)

• I need to override the text spoken by TalkBack when a TextInputLayout/TextInputEditText combo is read (done) and when it gains focus and the contents are read again for edition (couldn't make this work, delegate is ignored).
• I've tested on:
  • Samsung S20 (TalkBack)
  • OnePlus 9 Android 11 (TalkBack)
  • API 31 Emulator with (TalkBack compiled by me from source) as described there. Then dragged TalkBack APK to the emulator.
• In most cases they all behave the same, except some Samsung Stupidity™.

Longer Question With Details

I've created a blank Android project in the latest Android Studio as of April 2022 (Bumblebee). No Compose UI, nothing special. Min API 28 to reduce variables. Just an empty activity.

I enabled ViewBinding (but it made no difference), and made sure the latest stable dependencies are set (no "warnings" of newer versions available).

    implementation 'androidx.core:core-ktx:1.7.0'
    implementation 'androidx.appcompat:appcompat:1.4.1'
    implementation 'com.google.android.material:material:1.5.0'
    implementation 'androidx.constraintlayout:constraintlayout:2.1.3'

Layout

The Layout for MainActivity is quite simple. A root ConstraintLayout and a (useless) LinearLayout in vertical orientation to quickly stack views for testing, but I'll use only one child (in practice there may be more, but removing this ViewGroup made no difference, so I left it).

Here's the "rest" of the layout (removed the wrapping LinearLayout attributes for brevity):

    <LinearLayout ...

        <com.google.android.material.textfield.TextInputLayout
            android:id="@+id/inputLayout"
            style="@style/Widget.MaterialComponents.TextInputLayout.OutlinedBox"
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:hint="Phone Number"
            app:endIconMode="clear_text">

            <com.google.android.material.textfield.TextInputEditText
                android:id="@+id/inputEditText"
                android:layout_width="match_parent"
                android:layout_height="wrap_content"
                android:inputType="phone" />

        </com.google.android.material.textfield.TextInputLayout>

    </LinearLayout>

This looks like you'd imagine. An outlined EditText (in Google Purple, the default primary color in the theme). The IME is set to Phone.

InputType

As you can see, I've set the inputtype to phone as I am interested in a keyboard for Phone Numbers. This offers a few small (but potentially useful) features on the default Android Keyboard for entering Phone Numbers (and not just digits).

In practice, setting this to number or phone made no difference in the problem outlined in this question.

MainActivity?

The Activity is empty, and only the code I added in onCreate exists.

 private lateinit var binding: ActivityMainBinding

 override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        binding = ActivityMainBinding.inflate(layoutInflater)
        val view = binding.root
        setContentView(view)

        // Set a phone number value
        binding.inputEditText.setText("628123123") //this is valid in the NL
 }

TalkBack "Issue #1"

Running the above, looks like you'd expect:

If you enable TalkBack and read this, it will be read as "six hundred twenty eight million..." and not as "Six, Two, Eight, One, Two...". This is very silly on the default TalkBack behavior, as the EditText has an input type of Phone, it should use that as a default to define the actions/events and how the Accessibility Nodes are created. It doesn't

But there's a way to workaround this. BYOAD™: Bring Your Own Accessibility Delegate.

All right.

A Custom Accessibility Delegate

class MyAccessibilityDelegate(private val layout: TextInputLayout):
 TextInputLayout.AccessibilityDelegate(layout) {

    override fun onInitializeAccessibilityNodeInfo(host: View, info: AccessibilityNodeInfoCompat) {
        super.onInitializeAccessibilityNodeInfo(host, info)

        layout.editText?.let {
            val content = it.text
            info.text = content.toPhoneNumberContentDescription() // More on this below
        }
    }
}

And you set this in the actual InputLayout, not the EditText:

binding.inputLayout.setTextInputAccessibilityDelegate(
            MyAccessibilityDelegate(binding.inputLayout)
        )

Wait a moment, what is toPhoneNumberContentDescription()?

Just a function to transform a phone into something more readable. I'm not going to provide the code because it's irrelevant, but it looks like:

fun CharSequence?.toPhoneNumberContentDescription() = ...

What it does is separates this: 628123123 into 6 2 8 1 , 2 3 1 2 , 3

TalkBack reads the digits with a better cadence and pause; you can format this anyway you want, this is irrelevant to the next TalkBack issue.

TalkBack Issue #2: Focus.

The above Delegate solved the issue with TalkBack reading a phone number as a single number instead of individual digits.

If you're familiar with TalkBack (and if you aren't then you should), you can "double tap to activate/edit/etc". So if you have focus on this EditText, you can double tap anywhere on the screen to "click" on it to let the EditText gain focus and let the IME trigger the keyboard and all the [mostly broken] magic that android does behind the scenes to show your keyboard and set a view as focused.

It also triggers the View's accessibility, as TalkBack is enabled, it now informs the delegate that a new event (accessibility focused) has been triggered (among other events) and TalkBack reads the contents alongside some of its own additions of "edit text, blah blah".

The issue is that after the double tap, TalkBack reads the numbers again as a single big number, not the modified version provided by the above delegate.

This means it reads "six hundred twenty eight million... showing keyboard"

If you the swipe with two fingers to go next, you hear the "clear button" description, if you swipe back, then it reads the EditText again, but this time, it correctly reads: "Editing, six, two, eight.. edit box phone number..." so clearly the "double tap to activate" event/Action is triggering TalkBack to read from the input text directly, ignoring the Delegate that is supposed to format this.

What I couldn't figure out

I may be tired and lacking more coffee, but I couldn't figure out how to override this event. The other methods in the Delegate receive the events which contain the text to be spoken, but the text in that event is read only so I must be missing where/how to set this up.

Does anybody have experience with this that can give me pointers to it?

Fun Not So Fun Fact

If you prefix any number of digits with a 0, it will be read as individual digits: 0628123123 is correctly read as zero, six, two, eight, one... etc.