Better Testing - reward players for helping

[Torquemarda]

watching daily chat, especially around releases it is clear how many people get frustrated that 5 minutes into an event or new functionality we see things not working and a emergency fix is scheduled, servers go down, game time lost etc. We all now how it goes.
It's clear that insufficient testing occurs, and given BB's relative size it's not surprising as it's tough to test every possible combination.

One real problem for me is when new stuff comes out I don't want to go to the test server and spend hours buying generals with the very generous amount of gems etc., then doing the skills to get the generals ready and then play some adventures. We just want to get on and play.

So why not harness the power of the community, there's a lot of loyalty amongst the players, I for one have played for many years so we clearly love the game.

I'd like to suggest that BB think about the following:
Testing - When logging onto test they have two choices
1. BB Setup - as is, use island provided by BB for testing purposes
2. Clone your island but also have sufficient gems/resources to play around and test out all the new cool stuff, but not as many as option 1.

Secondly, the more you play the greater the rewards, so if we choose option 2 from above then 50% of the XP earned from combat is transferred back to our main island.
If you choose option 1 then a gem reward proportionate to the time played, and actually played not time browser is open.

Let's take the recent Observatory fiasco. It's been out for ages, still doesn't work, despite at least one attempt to fix it. Why is it still like this? Because when it was first released into test no one did the skills on the generals that caused the endless loop, no one sent those generals to adventures. Why? because we're not testers, we're not incentivised to test properly.

I go onto test, take a quick look at new features, buy the buildings etc. check placements, space etc. Minimal amount of time possible before going back to what I love - playing my settlers!

Clearly you'd need to think how to balance the XP reward with the huge resources available otherwise we'd be doing endless Mountain Clan or similar and then that wouldn't be balanced, then again you'd not be getting the resources back to your main island either.

I don't think the above is a perfect solution, but I think it's a lot better than what we have now :D

[Mannerheim]

The issue is that for example both the Christmas tree and Observatory issues were clearly reported already during test phase, weeks before hitting the live servers. The same applies to many other issues. The reward system would not help anything if the issues are not even being looked at even with the current system. It's even possible that reward system would just cause more harm as people would go after rewards and begin to report everything, even old issues and create so much "noise" that the relevant issues to the latest changes would be lost.

[Harlequin_1963]

I was told by a member of support that test and live are two totally separate systems and that even if it works on test it might not work live..
( words altered from original twitter answer as not allowed to pass on messages from any BB staff )

[Torquemarda]

thats even worse then Mannerheim than I'd realised - so just poor quality control and a lack of respect for the customer/players

[BB_Torkav]

I'd just like to point out that the Christmas Tree has been much improved with the [Change Log] 19/03/19 (https://forum.thesettlersonline.com/threads/36856-Change-Log-19-03-19) and the Observatory is almost fully functional now (only the Quartermaster Generals need a fix).

[Norton_C]

I'd just like to point out that the Christmas Tree has been much improved with the [Change Log] 19/03/19 (https://forum.thesettlersonline.com/threads/36856-Change-Log-19-03-19) and the Observatory is almost fully functional now (only the Quartermaster Generals need a fix).

Sorry, but "almost fully functional" in this case is almost as good as completely broken cause if my troops get stuck in QM landing loop what good is it that my other generals arrived earlier?

[Kriegor]

The issue is that for example both the Christmas tree and Observatory issues were clearly reported already during test phase, weeks before hitting the live servers.

And the issue is easy to recreate. So testing was never needed from us, it was childsplay to recreate.

And still it got released.

[Nogbad]

Afraid we used to get rewards for testing in the past, but it didn't make any difference to the bugs going live, and at the time BB seemed more bothered about "stress testing", ie: performance hits as number of players increased rather than what the new content did to the game.
Then during 1 particular "reign", bug reports went unanswered and then batch moved to the "No Bug" section at the end of that test phase completely unaddressed and horrible things went live at an alarming rate. Really undermined the whole point of TSOTesting, it just became a playground for those who couldn't/wouldn't buy gems in the real game, even worse just playing on Test left stuff in cache which then interfered with the smooth running on the live servers, I ended up playing it in a sandbox until giving it up as a bad job.

[Torquemarda]

Appreciate that Torkav, and I do admit it's a huge improvement. Thats what I love about you guys, we can talk to you and you do take on board the comments.

[sparkz]

It's clear that insufficient testing occurs, and given BB's relative size it's not surprising as it's tough to test every possible combination.

If a software company employs competent people, those people should be aware of testing methodologies and should not resort to testing every possible combination.

From the technical perspective vast majority of this games functionality is not even complex, meaning that poor testing can only be explained by deliberate negligence, not insufficient resources.


So why not harness the power of the community

Because its irrational to expect someone whos unpaid, has no prior experience and is not bound by anything like a contract obligation to do it PROPERLY.

__
Another thing is what people probably dont know and in the era of all those early access versions of the game might very well not expect it : its not customers obligation to assure the product is of proper quality, that obligation lies on the company that is trying to deliver it and expects to make money out of it.

Any decent company only includes some customers in the testing when the product is done, working properly and they are going to verify if it matches customers expectation (and possibly do some last minute changes in that regard), in the big software world involving customers in primary testing is a notion as ridiculous as involving real drivers in car crash tests.

[lordloocan]

I'd just like to point out that the Christmas Tree has been much improved with the [Change Log] 19/03/19 (https://forum.thesettlersonline.com/threads/36856-Change-Log-19-03-19) and the Observatory is almost fully functional now (only the Quartermaster Generals need a fix).

Mechanic to customer: Your car is ready sir, it's almost fully functional, just needs some tyres...

Shop owner to customer: Your vacuum cleaner is almost fully functional, just needs a motor.

'Almost fully functional' means nothing, it should function correctly.

[cliff60]

its not just BB with these probs. just about every game wether its browser or PC has bugs in that never seem to get fixed.

Ive bought games on steam that had so many bugs in they were unplayable. But was never able to get my money back and the devs just scrapped the game and went onto a new game without fixing the older game they sold.
Some devs r under presurre from steam just to knock out games as fast as possible without checking to see if they work 1st or not.
In the end its all about making money rather than making sure games work properly.

The CEO of 1 browser game i used to play did a press release saying he didnt give a stuff if the games worked or not he only wanted to get as much money out of players as possible before scrapping the game. Needless to say i quit that game when i saw it as did most of the other players.

There is 1 or 2 browser game devs that r always fixing games as soon as probs get reported and they have a lot of loyal players cause they work with the players to fix probs as soon as they get reported. But very few do that.