Here are some ideas for placing the Love Garden you can check out before you start to move your buildings.
If you have 1 Love garden, you should have at least 15 pop buildings around it, because it takes 15 to activate the first zone buff called "Heart Lockets level 1". There are several places were you can place 15 buildings of 2x2 size around the garden. If you have a second love garden you only need to have 5 or 10 buildings around it because there is no extra benefit from having more than 25 in total.
There are only 24 2x2 spots around the Love Garden that can activate the zone buff so for 25 and +300% settler growth you need 2 Gardens.
Here are the pictures:
1) Sector 3, 15 pop buildings. Maximum 19 (replace 3 wheat fields and one in the top middle corner of the grid)
2) Sector 3, 19 pop buildings is max. 3 field spots also included.
3) Sector 6, 15 pop buildings. Witch tower for example. bottom square of 2x2 buidling is the important one.
4) Sector 6, 17 pop buildings. move garden one more spot to NW for 18 spots
5) Sector 7, 15 pop buildings. This one needs a small rock removed where the Nords Residens is placed. Rocks can be removed with gems but is expensive. I decided to do this aanyway to keep the big areas in S6 and S9 open for future area buff buildings.
6) Sector 9, 17 pop building incl. improved storage spot
7) Sector 1, 20 buildings. Maximum is 24 but no point in that.
image links is fixed.