I concur that something is up with the counts. Quick spreadsheet work tells me:
On Adventure: 1645
Barracks: 750
Elite Barracks: 150
Unassigned Troops: 6571
Working: 311
Unemployed: 2023
Total Settlers: 11450
But current population count according to the Settlers toolbar is 11170 which is a difference of 280.
If it's my math please correct me, but the numbers are off from what I can see. I'm curious to see when the adventure is over and all troops are home if things add up which would mean that first-aid troops don't come off the population counter while on the adventure.