Units don't obey commands at times?

Units seem to "forget" commands and will either just stand still or attack an enemy close to them.

Am I imagining this? Every game it seems to me like I lose cavalry because they ignore my commands and instead attack spears/archers.