When looking at the QA team work load, roughly what is the split in testing between: Running through pre-defined scripts/regression testing. Adhoc/experimental testing. Targeted/specific feature testing (I.E gathering stats on ship performance). As a follow-on, what determines/changes this allocation (I.E what determines if you need to increase time on regression or spend more time on ad-hoc testing, etc)?asked by Logical Chimp
TL;DRQA time is a constantly shifting mix of regression, feature-targeted and ad-hoc testing, roughly given as percentages, and new blockers or fixes can reprioritise it from hour to hour.
Well, as we've mentioned, it's largely a moving target. You know, usually we have our set predetermined tests that we're going to do on every single build be it a smoke or a sandy check or something like that, and whenever a new feature is developed of course we're going to go through a gambit of tests that are designed around that feature, but we're also going to ad-hoc areas around that feature, you know as it comes out. I would say it's probably about 40% regression, 40% targetted, 30% ad-hoc, because you know QA, we give 110% every time.
110%.
Yes, that's right.
So actually we were talking about this earlier, cause like whenever bugs come in and, so we have a little thread in production that's like hey these are being flared out, and it's mostly just production and QA that are on these threads, and it's like hey, this is the latest blocker, this is the latest thing, whether it's found through the community or whether it's found just through us internally testing and all that stuff, these are getting major priority, so as soon as these are sent out the developers get it, and once the developers fix it, a producer's basically hitting up one of the leads like you saying hey, we just fixed this, here's the change list...
Lets get some eyes on it, let's get some bodies on it...
Exactly, and then people will just change focus like that and just be like alright, we need this, we need this now, we need this now, so yeah, it's like building in quicksand...
Pretty much, the situation is always fluid, from one day to the next, from one hour to the next, cause all of a sudden someone will come along with something new, and that all of a sudden takes priority and everything else that you had gets shifted down a rung on the ladder and you have to get to it later.
Yeah, it's always a lot of fun.
Fun is not quite the... no actually it is fun.
It's a lot of fun.
It's a lot of fun.