walking/runing uphill takes a lot of stamina, combine that with desert and yeah your units tire quickly
recovery is existent but you must rest your troops before they become completely exhausted
there's no minimod for this afaik as nobody seems to find it an issue but if you wanna change things yourself edit export_descr_unit file inside the data folder
Code:
; stat_heat Extra fatigue suffered by the unit in hot climates
;
; stat_ground Combat modifiers on different ground types. From left to right
; scrub, sand, forest, snow