Why not have a building which allow to reduce military labor,

Instead 1:1 ratio from a settler train into soldier, this building give buff to access 1:2,1:3,1:4 ratio of labor reduction per level of building upgrade,

There's time duration for this buff, about 1-2 days,

area of this building about 2x2.