Both towers must be destroyed before you send last attack to castle. If you send your last general when those towers are still standing and those towers are destroyed before your general arrive in to the area, general still use same route what was, when you sended general to attack. (Route is never changed, even camps are destroyed from path. Your general never do shortcuts.)
So both towers must be destroyed before sending last attack. I allways make sure that route is safe. I wait, that my generals that I ordered destroy those towers comes back to theyre garrisons.