Welcome, Joe!
It seems like you are learning for all the right reasons. Your father would be proud.
You could definitely go to Japan and work as an English teacher; I have several friends in Montreal who have done this (although some of them taught French). One of them even went to Japan without a job and found a teaching job once he got there.
Even if it’s not what you want to do for the rest of you life – if you feel like doing it – then I think you should go for it. No matter what happens, you’ll learn something valuable and maybe you’ll find or learn something on your journey that points you in the right direction.
The most important thing is to do whatever makes you happy and think positive. Don’t stop learning (that goes for anything, not just Japanese) and don’t let people put you off or drag you down. Don’t be afraid to ask for help if you need it. You can do it!
がんばって!
~ Leif