The first time you upgrade Barrack and Provision House, it almost halves the time to make the stuff. Second upgrade (to lvl 3) is somewhat nice and noticeable. But the 2 last upgrades are not only insanely expensive (as they should be, mind), but the benefit they provide is next to pointless.
It can best be descripted like this:
First upgrade - removes hours
Second upgrade - removes half hours
Third upgrades - remove minutes
Forth upgrade - remove seconds
Big level 50 players usually dont even have a lvl 5 Barrack or Provision House, simply because there's no point in it.
My suggestion would be that each upgrade removes 20% that's cumulative with each other. So at lvl 3 the Barrack trains recruits at 60% of the normal speed. And at lvl 5 it trains them at 20% the normal time.
The value I've mentioned is simply just to illustrate how my idea works. But this way its incredibly attractive for all players to upgrade to lvl 5 as soon as possible.