function with narrowed return type

Question

See complete example in the TS playground here.

I have two basic types for the model of my application that accesses raw data: Accessor and IndexedAccessor<A extends Accessor>. They all have read() functions that can return a particular primitive type (number and string for now). For this reason they are a discriminated union instead of subclasses. I'm trying to write code that is able to map model objects (aggregates of accessors) to DTO objects (same properties, but with the raw values coming from accessors).

export type AccessedType<A> = A extends IndexedAccessor<infer T> ? ReturnType<T["read"]> :
                              A extends Accessor ? ReturnType<A["read"]> : never;
export function mapToAccessed<A extends Accessor, I extends A | IndexedAccessor<A>>(a: I): AccessedType<I> {
        return a.read();
    }

Typescript issues the following error: Type 'string | number | ReturnType<A["read"]>' is not assignable to type 'AccessedType<I>'. Type 'string' is not assignable to type 'AccessedType<I>'.ts(2322)

I tried putting square brackets in the conditional parts of AccessedType, or just making the return type of mapToAccessed be ReturnType<I["read"]>, but none of these helped.

Can someone please show me how to formulate this properly, so that I can get such a generic static function, that takes an accessor of some type, and can return the corresponding type? If possible, I'd like to avoid coupling with the set of accessors in the sense that I don't want to write if statements, switch cases for something which is already conceptually typed.

On the flip side, could it be that the type system is lacking here? Could TS be made smarter in this manner? If not, where is my logical pitfall?