Why not have a building which allow to reduce military labor,
Instead 1:1 ratio from a settler train into soldier, this building give buff to access 1:2,1:3,1:4 ratio of labor reduction per level of building upgrade,
There's time duration for this buff, about 1-2 days,
area of this building about 2x2.