What is the name of the type of program to produce Unicode characters from ASCII combinations?

I believe a keystroke composer or a keystroke translator would probably be appropriate terms for any utility which converts multiple keystrokes into individual glyphs not represented directly on the keyboard.

Most likely this would be the keyboard driver. Your operating system will try to find out what physical keyboard you have (that is what keys in what place). The user can usually choose the keyboard layout, that is the rules that are used to translate pressing keys into characters, which is very much language dependent.

Any reasonable software will not check which keys are pressed on the keyboard, but which characters are produced by the keyboard driver. The keyboard driver is usually combined with some other software that allows entering any unicode character, not necessarily using the keyboard.

The general operation to transform some characters by others is a substitution. The more specific operation to transform an input message in an output of codes (such as Unicode) is called encoding. This would make your programme an encoder.

But in the end, if this is done directly at keyboard level (i.e.: you type oo and see ô) you certainly speak of the keyboard driver. Keyboard drivers for example catch dead keys to combine them with the next key.