Solved the problem yet?

It’s hard to list all possible solutions here, since I have no idea what your settings look like or what you did while uninstalling English keyboard layout (just an example – it’s impossible to uninstall keyboard layout while it’s set as default, it will just keep coming back <- maybe this happened?).

Anyway, as a quick solution until you find something better, why don't you just change key settings for language inputs? Instead of using alt+shift to toggle between inputs, create a new key combo for French input and another one for Japanese. EN might still show up in your Langauge bar, but you won't accidentally end up using it.