This falls into a more general, "controlled auto-attack" feature that has been suggested. The ability to set different limits on auto-attack (stop when the attacking troops reach a certain number, or when the defending troops reach a certain number, or when the difference reaches a certain number, or when the odds reach a certain level, ....) has been proposed before: DIfferent Levels of Auto-Attack
, now found in the Archived Rejected Suggestions forum. Still, the ideas there might be worth reexamining and developing into an add-on.
I'm not sure if it is still true, but apparently the Clickable Maps add-on had a way of controlling the number of armies you want to have left.