I’ve answered this question numerous times before, but this time I’m not going to mention the rules.

One of the biggest mistakes a beginner can make regarding kanji is to think that everything will be okay if he just memorizes the rules. There are only general rules, and those rules hardly matter when you actually start reading.

Each kanji has its own circumstances regarding what reading should be used, so such readings have to be understood from the meaning of the vocabulary. Case in point, you have understood that numbers like to use the on’yomi reading when alone, and the reason for that is that most of their kun’yomi readings have okurigana that constitute native numeric counting methods. Some other kanji might not have okurigana in their kun’yomi readings, but they may also have on’yomi readings that create a different meaning if used.

My point is, the key to reading kanji is to understand how a certain reading leads to change regarding vocabulary usage. The rules might help as training wheels, but they won’t benefit you as much when you start reading. It is still important to memorize the readings of the kanji, but memorizing vocabulary readings and seeing how those readings lead to a certain meaning is the best way to read efficiently.