Amount of settlers do not decrease after I tear down residences

I had built a couple of residences, and soon my amount of settlers had reached 60, total population 90. Then I teared two residences down, which would logically reduce my number of settlers to only 40, total population 70. But it does not. My amount of settlers is still 60, and total population is 90. In the Mayor's house, my population shows as 90/70.

Is this really working as designed?

How to reproduce:

Build a residence. Wait until settlers have reached max value.
Tear down the residence.
Population will not decrease.

I'm guessing all the extra settlers wouldn't just commit suicide after losing their homes... they'll die in battle at some point and not be replaced

Of course they would not commit suicide. But they are settlers. They would move on. They are coming from somewhere, so obviously there is at least one place that they can go to. It is not logical that you can exploit the game by building lots of residences, up your population, tear these buildings down (thus redeeming building licences as well as resources), and get to keep the settlers. You can both eat the cake and keep it.

you wont gain anything by doing it either, If you keep rebuild and destroy, you'll lose more material that you might need it later on.
the amount of workers is nothing compared amount of troops you need.
it started with 1, 10, and when you realize it, you need to zerg it.

You'll soon lose those 90 settlers then what?, you'll have to re build your houses for them , in the long run you will lose trying this.

This is not the point. Even though it is not exploitable, it is still a bug IMO. If I remove the houses, I should lose the population living there. Simple. The alternative makes no sense.