When the buff is active its meant to double the production of all active buffs on work yards this includes epic buildings and it does unless you change the production of the building then you lose the effect..it shouldnt happen as regardless of whats being made the buff is still active on the workyard