It’s been a long while since our last post. We are busy getting ready for another busy summer of QA. More posts coming soon.
It’s been a long while since our last post. We are busy getting ready for another busy summer of QA. More posts coming soon.
It’s pretty sad, but another truth of the quality assurance testing industry is that it’s cyclical. Crunch time, followed by the deathly quiet after the layoffs.
The tricky thing is to be one of the testers who does not get called into that uncomfortable meeting with the boss on a Friday afternoon. How do you keep your job when the job is no longer there? It’s a curious thing. Even when times are slow, certain testers somehow survive the swing of the axe. How can you increase your chances of being one of those survivors?
I should not have to write this paragraph, because it’s about painfully obvious stuff that “kids today” don’t seem to understand. But kids… if you do these things, you will be about fifty times better than most of the people in the work force today:
1. Show up on time. If you say you’ll be here at 8:00, be here no later than 8:00. Not too hard to figure that one out.
2. Take pride in your work, and do your job well, even if you hate it for a while. If you’ve read any of our other posts, you know that we often have to test games that we would not play of our own free will… Bratgirl’s Hairstyle Adventures, and the like. But if you’re assigned to test Bratgirl, suck it up and test it just as you would the latest version of Modern Warfare.
3. Communicate like a grown-up. In work communication, don’t use phrases like LOL or pwned or WTF. Write it all out like a big boy. Use proper punctuation and spelling.
4. Ask lots of questions and be sure you understand what is expected of you. Then, go do it.
Become an expert
Small companies love people who can wear many hats. If you are a tester who can also put a computer together, or write code, or set up a simple bug database, or make Excel churn out fancy charts, or write a blog (ha ha, nice try, pal. This is my gig), chances are you’ll be kept around during layoffs. For example, if your company is testing lots of Nintendo games, and you are the in-house NOA standards expert, you will probably be around for a while. It may take some extra hours and certainly some extra effort on your part, but it will benefit you in the long run. So, find a way to make yourself indispensable, and you’ll have a much better chance of becoming an old-timer.
Be the helpful one
The boss often asks for volunteers to help, and by “help” he or she may mean, “Stay late tonight and work on your game” or “Come in this weekend and help us move to a new office” or “Come help me load my friend’s moving van.” These extra credit projects are optional. You won’t be fired if you don’t do them. But to put it in gamer terms, it puts you higher up on the Leaderboards!
Sweet Jesus. I have turned into a nerd! Next thing you know, I’ll be blogging about company cosplay competitions…
If you are the arrogant one who is always telling the person at the next desk how to write up their bugs, or harassing the womenfolk by staring at their body parts, or if you drink the last of the half and half in the company refrigerator… Well, you might find yourself on the way out the door next time it opens wide. You don’t need to smooch anyone’s behind. Just be a considerate, hard-working person who shows up on time and doesn’t complain all day long.
There is no real secret to surviving layoffs. Just use common sense and good manners. We do what we can to keep the good testers around.
We are getting ready for E3! There should be some great games on display, to keep up with all of the blockbuster movies coming out this summer. Avengers, Dark Knight, Expendables… So much good stuff!
We are entering the busy season for quality assurance testing, when the $60 light bulb goes off and video game developers realize that they are going to miss their ship dates if they don’t start putting in some long hours and getting their games done. The traditional console game business is cyclical. Winters are generally quiet. Then things pick up a bit in spring. And all of a sudden, it’s summer, and everyone is in freak-out panic mode because they need to finish the game by August to get into stores for the big holiday shopping sprees.
What does this mean for QA testers? It means work, work and more work. Surprise!
When developers work overtime, they usually want their QA working, too. They often keep the QA team waiting while they fix one more bug. Once that bug is fixed, they hit the “SEND” button in their FTP program, and run out the door. When they come to work the next day, they expect a full regression report. So QA works all night.
It can be tough. What are some good ways to get through it?
CLEAR YOUR CALENDAR. Now is not a good time to enroll in a night class in Balinese shadow puppetry. There is going to be overtime. It has always been this way in game development, and thus shall it ever be. I don’t care how many two week Agile sprints someone has entered into a MS Project scheduling tool. Nothing is going to eliminate this ugly reality of software development.
RAISE YOUR HAND. Crunch time means the boss is going to ask, “Who can work late tonight?” And if you are a tester, you should try to be Roger, that person who always says, “Yup, I’ll be here!” The person who volunteers to help is seen by the boss as a solution to a problem. Bosses love solutions! ”Terrific!” says the boss to himself. “I can always count on good, old Roger. Awesome. Now I can go worry about something else.”
What bosses don’t love are more problems. If you can’t be at work testing on your game when it goes into crunch time, you have created a problem for your boss. And if you’ve been reading these blogs, you know we don’t see testers as just a warm body clicking on a mouse. You are an indispensible part of your test team, and you are NEEDED. It sounds harsh (and it is harsh), but understand that if you can’t work overtime, your boss might need to solve his problem by hiring someone else.
This is often very difficult. We have friends and families, and we want to have interests and activities outside the office. Even bosses have lives outside the office. Imagine that! But in the game industry, and ESPECIALLY in the state of the industry today, clients and customers are not tolerant of anything. They know they can take their business elsewhere, and sometimes, they do. Just like that. Sayonara. And if that occurs… Well… On the bright side, you can explore your passion for Balinese puppetry, or go design that WOW killer you and your friends dreamt up in game design school. You will have time on your hands.
But you won’t have a job!
It’s not personal, it’s just money. Harsh harsh harsh. Oh, the evils of money. Bad, bad money, greedy evil companies, money is yucky and evil and Balinese puppetry is ethereal and good.
But guys. And girls. I don’t know any testers who show up at work solely because of the raucous ribaldry of the test pit and the tasty Red Vines and those oh-so-comfy chairs. Testers show up because they are paid. Hey, wait a sec. All this talk about money reminds me…
CASH IT IN, BABY. The good part of overtime is that you get paid more. Sometimes, a lot more. Yay! Many line testers make more than lead testers because of overtime pay. Mythical Roger, that happy OT guy with the helium hands, probably drives a nicer car than his boss who is exempt from overtime pay. OT is painful, but it can be lucrative.
So, suck it up and make hay while the sun shines. Then, read my next blog entry… (cue ominous music) How to stay employed when crunch time is over!
The video game industry is undergoing some challenging transitions (that’s about the nicest turn of phrase we can muster). QA testing groups must adapt to the transitions as well. We are seeing smaller testing teams for longer periods of time on the games we’re currently testing. This is great in a way, because our testers get very familiar with the dev team and the project. When our client is pushing new updates of a game every week or two, we can continually refine and improve our processes.
But after a while, even the most die hard tester needs a little change of scenery. We like to keep them happy, because a happy tester is a… well, a happy tester. But we have a few big, beefy men who have been testing ”Rainbow Butterfly Hair Adventures” for a few months. Months. They would give their left arm to work on a first person shooter. But that would make the control scheme a little tricky. We generally frown on that.
So, we prefer to rotate our testers from time to time to get fresh eyes on a game. This ensures that the same old, same old way of testing a game won’t bypass a nasty bug. Plus, we like to spread the wealth of knowledge and experience around. Some handy time-saving technique that works on Rainbow Butterfly might benefit another title. If everyone has their noses buried in one game, every day, that experience is never shared.
Of course, we don’t drop fresh testers onto established games without support and training. We typically have them “shadow” the established team, and then shift onto the project during a lull. In this business, there aren’t a whole lot of lulls, so we have to be nimble, but it can be done and the benefits can be substantial.
The producers are usually the hardest people to convince about this technique. I’m a producer, and I know I always want my trained, trusted testers, forever. But it’s important to understand that fresh testers can help a product succeed. And besides, those trained, trusted big beefy guys aren’t going anywhere. They just slid their chair about 48 inches to the left of where they were yesterday. If you need them, they’ll be there.
We have lots of competition from overseas quality assurance groups. It’s understandable. Labor is cheaper, and with budgets tightening all over, everyone is looking to shave off some dollars. Quality assurance testing is a labor intensive phase of production, so from a budget cutter’s point of view, it’s low hanging fruit.
But before you get out the machete, keep in mind that there can be hidden costs to saving money.
I am going to let a little professional pride sneak into the post here, but this is absolutely true: Our testers are awesome! They were not just pulled in off the street and plunked down in front of a TV screen. They are not preschoolers sitting in front of a monitor, banging on the keyboard. Everyone has a bachelor’s degree. Some are programmers in their spare time. They have all been carefully screened and put through our training program. And if they don’t live up to our high standards, they are asked to leave.
We run a very tight operation, and we can’t afford to have any goof-offs. We understand that a smart, sharp tester can get to the bottom of a bug a lot faster than an inexperienced, cheap tester. That’s fewer hours, more efficiency, better testing, less money.
Producers who work on titles that are tested offshore often bear the brunt of the cost saving, because guess who gets to stay late to answer the off-hour emails, and get to the office early for phone calls, and generally manage yet another external group? The producer! Producers are known for juggling many tasks, but if you are working more hours on your project because it’s being tested overseas, are those hours documented in the budget anywhere? Most producers I know are working on more than one game. When the game you have in test overseas has problems, are you calculating the effect this has on your other projects?
Your audience is sitting in our office. Our testers play everything out there, and they buy lots of games. They will know if you are being competitive with the latest and greatest games, and if you want them to, they can make excellent suggestions for improving your game. It’s unlikely that a tester in India can provide the same feedback. If your game is not keeping up with the competition and sales are lower, is that lost revenue calculated anywhere?
If your sleep-deprived programmer has to read and re-read a convoluted bug translation, gets confused, sends a “please clarify” email, waits for that email translated and given to the tester, it gets sent back, translated, etc etc etc… Aren’t you perhaps WASTING SOME TIME? Wouldn’t it be nice to just pick up the phone and call one of our testers about the bug? We all speak pretty good English, and we love hearing from the dev team. Communication delays occur, but the costs of these delays are not documented anywhere. But you know they occur.
It may seem like a great idea to lower your QA budget by going to Asia or elsewhere for testing, and for some QA situations, it may make sense. But before you decide to take it elsewhere, try to add up the hidden costs.
As reported previously, we’ve expanded our video game testing service to include cell phone games… iPhones, iPads, and Android apps. We love these projects, in part because they don’t require huge amounts of expensive hardware.
But where do you get the hardware? It’s very easy when your game is a PlayStation 3 game. You just take a few wheelbarrows full of gold to Sony, and you get a test kit, same with Microsoft or Nintendo.
But what about an Android device? Who is the manufacturer? What is the official hardware spec? We have phones from:
And that’s not a complete list of manufacturers. There are all kinds of oddball manufacturers out there, as well as a few not-so-oddballs like Dell, Garmin, and so on. Each manufacturer has several devices… There are 24 for Motorola alone.
And each kernel version is a little different, and each browser is a little different.
What is an earnest and thorough QA group to do? Well, you need to get an assortment of phones, manufacturers, and operating systems.
You do get some new phones, and any day that you get to go shopping for a smart phone is a fun day. Nothing quite like tax deductible retail therapy! But you don’t want to limit your testing to brand new phones. You need to buy up a bunch of used phones, so that you are testing on what most people out there are using. So where do you go?
I confess to having an unhealthy obsession with Craig’s List. There are a lot of crazy people out there with crazy things to sell or give away, and I have to read about them into the wee hours of the night. I am intrigued. Just what is in that “Curb alert!” or “Garage sale leftovers!” Where can I dump dirt for free… What horrid grammar and spelling mistakes will be found in the listings? ”Swing has broke but I think it could be fix.”
Back on topic.
If you’re buying items on Craig’s list, use caution. There are about a bazillion people who buy things from people on Craig’s list with no problems at all. There have been precisely zero newspaper stories about me and the nice dining room table and chairs that I sold to a nice young couple a few years back. But there have been a few evildoers who get a lot of publicity. Be smart and safe.
Before you go off to someone’s evil lair to buy their old phone, a few tips:
If you meet someone at their home, you are less likely to be stood up, and less likely to be buying stolen property. Just be sure someone knows where you are going and when you’ll be back. Even better, go with a co-worker or friend who can wait in the car. And go during the day. I’ve also arranged to meet people at Starbucks, which is a good second choice. It’s a very public place, and they are everywhere. Not crazy about the coffee, but that’s just me.
Check model numbers before you go. Most manufacturers have some standard naming conventions, so you might save yourself a trip by getting the numbers first. Apple was quite sneaky with the iPod Touch. There was the original, then the second, third and now the fourth generation. Many people I contacted on Craig’s List thought they had a third generation, but by checking the model number, I found many that were second generation. It was a wonderful Sherlock Holmes moment to savor, but I actually needed the third generation. So I had to go right back to my computer to find another Craig’s list seller.
Know competitive prices. I hate to haggle. I settle on the price on the phone or by email. Then when you meet the seller, you just hand over the money and get the phone.
Get original packaging if possible. This reduces the likelihood that you are buying a stolen phone. You also get more useless ear buds to add to your collection.
Get a receipt. Remember the tax deductible retail therapy? The IRS gets quite irate if you claim expenses without receipts. My accountant gets even more irate. So, I have printed out the email communications I had with Craig’s list sellers where price is discussed, and I hope the IRS will accept that as a receipt if we get audited.
The ultimate best situation is, buying a phone from the mother of a spoiled teenager who had an iPod Touch, but just upgraded to an iPhone, and mom just wants some of her money back. Everyone wins.
A safer, more reliable source of used cell phones are used phone retail stores. You will pay more, but you’ll get a bona fide receipt for tax purposes and you’ll at least have some theoretical recourse if the phone you buy is busted. They often give a limited warranty. You can also get a bunch of phones at once, and save some driving-around time. Try to find one of these places near a college or university in a nicer part of town, so you have more inventory to choose from.
Enough already with the useful information. There are some new listings of “free empty egg cartons” and “dead vegetation (palm fronds)” that I need to read more about!
Offshore testing. There are a lot of groups in other countries who offer the same services that we offer at lower prices. We keep our rates as low as we possibly can, but even so, it’s tough to compete with countries that have a substantially lower cost of living than we do.
It is also difficult to compete against companies in countries that give tax breaks to high tech companies, because game testing is classified as a ‘high tech’ job.
So why use us?
In these tight economic times, it’s tough to justify the more expensive QA vendor. But if you’ve read this far, maybe you are considering it. So let me provide you with some ammo. While sneak peeking at sites of our competition, I came across this. Please note: This is under the “Quality Assurance” tab on their own web site.
A screenshot is worth a thousand words.
Videogame testers often get sent “on location” to test games at a developer’s office. It’s an expensive undertaking, and it usually occurs only when a game has a very aggressive release schedule. In these cases, the cost of sending testers on location can be justified by the time that will be saved.
Tight feedback loop. Imagine this scenario: A tester working at the QA office reports a crash bug that occurs after you beat the last boss in an 8 hour game. He carefully documents the fifteen steps needed to recreate this bug with 100% accuracy. Across the country at the developer, the programmer has this bug assigned to him. He reads the bug summary, but he is of the Twitter generation, so any string of text longer than 140 characters cannot be understood. Without carefully reading the steps, the programmer takes a quick peek at the game and tries to recreate the bug on his dev kit, but he leaves out a step, so the bug does not occur. He shrugs and says, “Not happening on my machine.” He opens up the bug database and marks the bug “fixed.”
When the next version is posted, the testers read that the bug is fixed. They get very excited, thinking a code change has occurred. Since the game is close to completion, the testers don’t have cheat codes. They have to play through the whole game just the way the end user does. So, they dutifully go through all eight hours of playing the game and every one of those 15 steps to duplicate the bug. And what do you know, it’s still there! So they get angry and assign it back to the programmer. The programmer gets angry, and again says that the bug is fixed. And on and on and on.
To be fair, it is quite possible that once or twice in the history of quality assurance testing, a tester might have missed a step when writing up a bug. Or possibly a tester didn’t write the steps with laser precision that we train our testers to use. I will grant that this is a possibly possible possibility. However, in the unlikely event that this occurs, the end result is the same. The programmer can’t reproduce the bug, and if he or she can’t reproduce it, it won’t get fixed.
But wait! If the tester is in the next room, he can drag the programmer to his desk and point at his screen and say, “Look. Here is the bug.” Or better yet, the tester can sit at the programmer’s desk and recreate the bug on the dev kit, so the debugger can catch it.
And when the tester gets this chance to demonstrate the bug, there it is… as plain as day, that ugly crash bug. Oh. I. See. A blanket of silence falls over the room. The programmer hangs his head in shame and slinks off to flog himself (but not until he changes that semicolon to a comma). Takes him all of 15 seconds. Multiply this by your bug count, and you can see how much time you can save by having QA testers work in the same office as the development team.
Working at a developer is a great way to learn about how games are actually made. You’ll see how the game development engine works, how builds are pulled together, how textures are changed and geometry of the environment gets tweaked. You’ll see wire frames of the game levels, collision boxes around characters, and data tables and animation engines and how to adjust camera… It’s like going to game development school without the student loans.
And the best benefit of working on location… you can expand your network of contacts, so in the future, when the developer is staffing up, and they need a junior level designer… they might just call that reliable, professional, smart and capable tester. Or they might refer you to someone who is hiring. You never know, so you’d better connect on LinkedIn.
More good things about going on location… if you are sent off site to test, that means you are critical to the team, respected by The Boss and deemed responsible enough to handle the pressure and stress away from home. And there will be stress. You are isolated, away from your family and friends. You tend to just work and sleep. When you do get a break, you tend to go a little bonkers, to relieve the stress. I’ve seen it with my own eyes. The first tester I went on location with ended up cleaning his barf out of our rental car after a night of “stress relief.” Yuck.
Be Prepared. Bring a laptop for email, because you might not get to use the developer computers for your personal email. Bring some books, some movies, things to entertain you during off hours. And pack some clothes. Seems a little obvious, but you never know. I went to Seattle with a tester once, and he didn’t pack any clothes. He just had the clothes on his back. I asked him, “You do realize that we are scheduled to be here for two weeks, don’t you?” And he said he did, but he just didn’t think he needed to bring clothes. To this day I don’t understand what he was expecting. Was he just going to wear the same shirt for 14 days straight? So, instead of going to the developer to report to work, we had to leave the airport and go to the mall to buy him some clothes. And then he got all picky, because the pants weren’t really quite right, not quite baggy enough. That’s when Nice Producer disappeared and Mean Producer came out and barked at him to shut the heck up and just buy some dang clothes.
Speaking of clothes, if you go test at a developer, be prepared to loosen your belt a little. The most frequent problem with testers on location is weight gain! One of the perqs of working at a developer can be free sodas and frozen burritos. Seems like a perq, but it’s really a just a clever way to keep you in the office a little longer. They usually offer this to the test team, too. The testers can get a little delirious about this, because we at DDF are notoriously frugal about everything. Stop breathing so much, people! Oxygen ain’t free you know! Wait. What? It is free? Hmm. Well, leave some for the rest of us!
And the hotels? Oh, man. They are losing money on us. Our hotel has free breakfast. And not some lame continental breakfast with bad pastries and a bowl of apples, but a real, full-on breakfast. Once our team checks out and the hotel gets wind of the breakfast bill, they might want to review this policy. Take a look at this order of bacon… for one guy.
Just a typical light meal before a day of work.
So, yeah. Back to the testing. It’s great to work so closely with developers. We love it, it’s exciting, and it’s a great opportunity for our testers to branch out a bit. But we miss our guys (and gal). Be good, be safe, and come back soon!
It’s not Thanksgiving yet, but I’m already in the mood.
Digital Dream Forge had a great summer. We were busy! In the past months, we have tested eight console games, two handheld games, two Facebook games, four iOS apps, six websites, a content management system (CMS) and numerous e-learning courses. Holy cow! We are proud to say that several of our games were approved on first submission and all of our iOS apps passed through Apple on their first submission.
And now that we’ve caught up on sleep and have gotten somewhat reacquainted with our families and friends, it’s time to give out some well-deserved THANK YOU’s to the many people who helped us succeed.
Mystery Man M, you inspired this whole thing back in November of 2009. Without your confidence in us, we would never have started on this venture. And since then, you have repeatedly recommended us to clients, introduced us to people, and supported us in so many ways. Kind of like the world’s best corporate Wing Man! We have said “thank you” many times, but those two words are not really adequate to express our gratitude.
Robert, Krysten, Liz, Amy, Chad, and Susan, you saved us so much time and effort, it’s hard to imagine how we could have recruited so many great employees without you.
And finally, THANK YOU to our testers! We live or die based on what you do every day. Of course, we paid you, and yes, we did bring you doughnuts from time to time, and let’s not forget that very special day with frozen dessert treats… But all kidding aside, please know that your efforts are hereby recognized and appreciated. You guys and gals rock!
Now… Let’s make the next few months as fantastic and busy as the last. We have some projects coming up that look like lots of fun, and we even have some irons in the development fire that could be very interesting.
Stay tuned for more updates, and remember to be thankful today for what you’ve got. No need to wait for November!