Why does TypeScript not infer the right type for this `const foo = {...} as const`?

I have a constant object (const pronouns = {...} as const) that models pronouns in some languages. Its shape is basically { [L in Language]: Record<PersonalForm<L>, PronounInfo> }, with Language being a simple string literal union, PersonalForm<L> being a language-specific subset of the template literal type TheoreticalPersonalForm, and PronounInfo holding the actual pronoun info. Instead of listing the allowed/needed personal forms per language explicitly, I define it through the shape of pronouns itself, like this:

type PersonalForm<L extends Language> = keyof ((typeof pronouns)[L])

The problem is, when I then try to access the PronounInfos for a particular PersonalForm elsewhere (in a function), TypeScript says Property 'direct' does not exist on type <blabla> ("blabla" standing in for a huge type expression):

function mysteriously_failing<L extends Lang>(lang: L, pronoun_p_f: PersonalForm<L>) { let pronoun = pronouns[lang][pronoun_p_f].direct // ^ error here }

Playground (note the type names I use are slightly different than in the example).

Why doesn't this work, what I can do apart from casting as PronounInfo?