BB Server infrastructure tour / info

[Meshaw]

Hi,

I'm a sucker for a good server room / data centre tour and love to learn "how stuff works" (queue the cheesy music!)

http://i.marktplaats.com/00/s/MjUwWDQwMA==/z/hDcAAOSwPhdU6MwV/$_85.JPG

I'm really interested to know what hardware & software set-up BB / UbiSoft use to provision the realms (Sandycove / Newfoundland etc).

Excuse my ignorance here - just firing out a few topics for discussion:

Hardware:

Is the back end for each realm comprised of lots of blades in a cluster or just one very powerful server?
Are the servers real tin or virtualised across a load balanced cluster?
Are the databases for all realms on a huge global SAN (for the sake of maintenance / scripting) or does each realm get it's own SAN etc? (I suspect the latter!)


Software:

Am I correct in saying the front end of the game is downloaded to your PC (in the form of a Flash package / executable which contains all the graphical images / animations required to play the game)?
The image of your island / adventure game-play are not actually rendered on the BB servers: the servers just send instructions to the game front-end as to there objects should appear and what they should be doing etc? (Flash renders the server's instructions locally on your PC?)


Obviously that is massively speculative - please set me straight!

I suspect the BB will never give us a glimpse, for the sake of security. So; calling all tech experts amongst us - do you think they do it?

Thanks in advance
Al.

[Qualan]

they are real tin foil :P

[Meshaw]

LOL thanks Qualan - they may as well be at the moment Lol.

I'm surprised how much maintenance is required. I know today's downtime is a special case because of the Easter event - but that raises another question; what do BB do in regular weekly maintenance?

Al.

[Qualan]

looks like it is software related- so guess their game engine might just be clunky and really would need complete redesign (which they probably wouldn't do from an expected return on that investment)

The weekly downtime was introduced way back when servers kept keeling over or running very slow when running too long

If that is the case- improved hardware over the years will have limited effect in the way the game runs really- although it would be interesting to know what they are running at the back end

Hardware improvements have had some effect over the years but the last year or 2 it seems to have not had much effect when they said theyw ere upgrading servers )

[Meshaw]

Ah I see.. when did the weekly maintenance start please?

I guess they were probably upgrading for expanded capacity rather than for increased performance.

The game generally runs pretty smoothly for me... I get the occasional day where things are a bit clunky but not often.
I'm at level 40 at the moment and my island isn't all that full.

I expect there are many factors that effect performance which are outside BB's control such as IP transit routes, your broadband speed contentions / congestion, the speed of your PC / browser etc.

Al.

[Peajay]

I have never known an online game to need maintenance as long and frequently as this one.

[lordloocan]

I thought there were lots of little people inside my computer and the delay was an industrial dispute. Front end, back end, this some sort of pantomime horse?

[Durin_d]

disclaimer: these are my findings and reasonings and thus can be completely wrong

This is what I've gathered from watching the network traffic from the browser with chrome developer tools.

Front end / game client is a flash binary that is run on our browser using Adobe Flash player plugin. This client is downloaded from Ubisoft CDN (Content distribution Network) and that network is also used to serve all the graphics and sounds.

The client talks to two back end systems one is for chat and one is for the game. The chat uses Jabber/XMPP over http and most likely the the software run on the server is Openfire. The game traffic uses amf (Action Message Format) to talk to the back end servers. There are more than one server to serve one realm. On Northisle I've noticed that there are 4 servers that serve home and adventure islands and the client can change the server when we visit friend or adventure. There is also one coordinator server that gives the client the address to use when visiting other islands. Servers that responds to the amf-traffic are using Jetty which is a web server software that provides Java web servlet engine so the server are coded in Java.

All the servers use the same IP-address so that makes me believe that there is a load balancing switch or server in front of all of the servers directing the traffic.

server addresses related to Northisle:
chat: w02chat01.thesettlersonline.com
coordinator: w02bb01.thesettlersonline.com
game server 1: w02g01.thesettlersonline.com
game server 2: w02g02.thesettlersonline.com
game server 3: w02g03.thesettlersonline.com
game server 4: w02g04.thesettlersonline.com

The servers used to be in UK Rackspace facility (based on IP-address) and are now In France in Ubisofts own facility. Since the servers used to be in Rackspace I believe they are virtual servers that can be run on almost any type of hardware.

[CoffeePowered]

most of the game data is downloaded by us for me its been around 700mb at times though that could be multiple instances but either way its alot of excess just for a browser flash game, the excessive maints who knows why a browser games needs so much maint it isnt like its a complicated game or a real mmorpg , maybe there are dupes or something that BB cant fix that players are exploiting and get rid of hence the maint/db work

who really knows you will never find the real answer in a forum or from game devs in all of my years of playing NO game dev has been truely honest and given direct reasons for anything they always talk to the player masses like they think you are an ignorant child

[Meshaw]

Many thanks for the excellent insight / suggestions Durlin and for your comments Coffee, much appreciated.

If Northside, for example, is provisioned by 4 servers which themselves exist in a virtual environment... does anybody have an idea what type or hardware would run that?

2 x very powerful machines each with running quad x hexa-core CPUs, 100's of GB or RAM and a central SAN? Running ESX? (presumably not MS / HyperVisor?)

I'm fascinated by this stuff :o)
Al

[Thejollyone]

they still run on Windows 3.1 :p

[matie2506]

oooh that was the most understandable text i seen here jolly :p

[Bluesavanah]

LOL thanks Qualan - they may as well be at the moment Lol.

I'm surprised how much maintenance is required. I know today's downtime is a special case because of the Easter event - but that raises another question; what do BB do in regular weekly maintenance?

Al.

I hate to be cynical but I think they feed all the bugs, this takes some considerable time and effort.

[FishSmell]

I hate to be cynical but I think they feed all the bugs, this takes some considerable time and effort.

Or maybe they hire pest control services weekly and it takes time until all the sprayed chemicals dry and it is safe to turn on anything electrical again. :D

[Nogbad]

http://thesettlersonlinewiki.com/uploads/178522219sinclair_zx80_SDIM0759.jpg

[Qualan]

they also have a baboon working as main engineer on the servers :P