How much automated regression testing are you able to implement versus inefficient and messy manual testing?
TL;DRAutomation is hard for this project and only slowly progressing; they have an embedded automation person and TestRunner, which does basic launch sanity pass/fail checks on builds, and they want to do more.
That's totally fine. I understand. Yeah, so actually, automating testing for a project like this is quite challenging, to say the least. As it stands, we are moving slowly towards being able to automate certain test cases. We do have a guy who is embedded automation, actually, and he is working slowly towards making it more available to all of us. But we do have a couple of things that we use internally already. George, on your time on the tech team, you might have touched on some of this stuff with TestRunner.
Yeah, that's correct. So TestRunner would grab a build and run through basic sanity checks. It doesn't go too in-depth, because obviously when you're making an internal development build and then running it through TestRunner, it can take a few hours to do that. So it goes through the basic checks. And when we try and grab a build, it'll give us all the information: this was fine, this was fine, this was fine, but this was broken. Sometimes we can get around it because it's not a core thing that would stop us from testing. But there have been cases where we tried to grab builds and it would just break continuously. That's where TestRunner picks things out, so we can identify that before we go into game, to say, we need to do this task, but this build is broken, what do we do? And then usually the devs will go and fix it ASAP and kick a new build, or we just revert back and then carry on the testing.
Yeah, exactly. It's mostly launch sanity checks and that kind of thing. Very basic pass/fails at the moment. But we are working our way towards it, and it is something that we are very interested in. So if you are an authority on automated testing, please do come in. We would love to hear your opinions.