Tools Team Test – 5 steps to better tools

This post originally appeared on http://altdevblogaday.com if you have comments that would be the best place for them.

Several years ago Joel Spolsky wrote about The Joel Test. A simple yes/no test you can use to measure the quality of a software team.

It looked like this:

  1. Do you use source control?
  2. Can you make a build in one step?
  3. Do you make daily builds?
  4. Do you have a bug database?
  5. Do you fix bugs before writing new code?
  6. Do you have an up-to-date schedule?
  7. Do you have a spec?
  8. Do programmers have quiet working conditions?
  9. Do you use the best tools money can buy?
  10. Do you have testers?
  11. Do new candidates write code during their interview?
  12. Do you do hallway usability testing?

If you haven’t tried it, wait until you’ve finished reading this post and then run through it for your team. Ideally you should be scoring 9+.
Inspired by this test, I started thinking about simple yes/no questions that could be applied specifically to the tools department on a game team. I couldn’t get to 12 but I came up with 5 questions that I think a good tools team will answer yes to.

1. Do you have dedicated tools programmers?

This was probably the first thing I thought of and is hopefully redundant if you are answering questions about your ‘tools team’. However, it is not universally the case that a game team will have programmers who’s only job it is to design, write and maintain the tools that the rest of the team use.

The problem with overloading your other programmers with writing the tools for their systems is that they don’t necessarily have the skill set to do it and usually their heart isn’t in it.
AI programmers want to write AI, not UI.
By taking the tools implementation away from the system’s author you get a second technical mind looking that the interface into that system. The tools programmer can be the one to worry about whether it is better to expose a range of 0-65535 or simply Low/Medium/High.
Graphics programmers think that everyone knows how many photons it takes to generate a good light-map.
The mind of a tools programmer is a wonderful thing, thinking about things like work-flow and iteration speeds. These are not the same issues that plague the resident graphics guru.
Audio programmers shouldn’t be writing code to render orientated bounding boxes.
If you force a programmer who’s interest is in one area to write and support the tools needed to use their systems one of the areas will suffer, and it will likely be the tools.

2. Do you deliver your tools atomically?

Often you will find that support queries to the tools team fall into one of these two scenarios:

Artist: The tools are broken.
Tools Programmer: Have you got latest from perforce/svn/git/cvs/the network drive?
Artist: No.
Get latest
Tools Programmer: Fixed
Artist: I love you.

Or like this:

Artist: The tools are broken.
Tools Programmer: Have you got latest from perforce/svn/git/cvs/the network drive?
Artist: No.
Get latest
Tools Programmer: Ah, still broken… I’ll get back to you.
Artist: I hate you.

To fix this, you must remember one thing: Source Control is not a delivery mechanism. Even if you have labeled/tagged a release there is nothing to ensure that the end-user has synced all the files necessary. ‘Did you forget the dev/bin directory, what about etc/other/important? Oh, you definitely need that one’.

These queries can be a significant chunk of support requests and you can make them go away very simply. Deliver tools updates atomically. By atomically I generally mean via an installer with a version number. This has the added bonus that each release has a unique version number which means bugs can be tracked more easily.

This opens up another question which is: Should you force a tools update on users? On balance I think the answer to this is yes, you should. It is likely that your users will need the latest tools to create data compatible with the latest game build and besides, if Google can do this with an browser shipped to millions of people I’m sure we can manage with one small toolset. It does, however, probably require that you have nailed the next question with a yes.

3. Do you test tools releases?

Your users must trust your tools. If they don’t trust your tools they won’t use them. They will hack at xml files in notepad and bad mouth you behind your back and to your face.

The level of trust from your users can wax and wane and it can be earned and lost. It will start out high, your users are a nice bunch, you’ve told them you have a tool that will make their lives easier. Why shouldn’t they believe you? However, if your tool breaks, and they lose data, and then they have to stay late to re-do it the night before a milestone you will lose that trust. Quickly. It’s a lot harder to earn it back than you might think. Once that trust is gone the users will hack the xml files forever because ‘your tool just crashes all the time’.

If you can somehow make sure that your tool doesn’t crash, and doesn’t lose vital data, then your users will continue to trust the tools and they will continue to use them. The answer is testing your tools before you release them.

Testing software is hard and it is time-consuming. On PlayStation Home, testing the full tools suite would be several days work for several testers. Which is why you need to rely on automated testing as much as you can. If you haven’t heard of them, look up unit testing, functional testing, Test Driven Development, Continuous Integration and Build Machines. Pick and choose which solutions suit your needs but do use this stuff to bring the testing time down, and your trust levels up.

4. Can users submit bugs and feedback?

I’ll keep this one short. You have a bug database for your game, right? Then get one for your tools.

5. Can your tools be used without a manual?

Your tools probably don’t have a manual, and even if they did, the vast majority of users won’t read it before coming to you for help.

Even though you’ve added an awesome new feature the users may not know what it does, and they may not know when they should or shouldn’t be using it.

Some of things I’ve tried in the past on teams are:

  • Seminar-style lectures – where the tools team go over new features and take questions from the team
  • Documentation – i.e. a manual
  • Release Notes – a list of new features with descriptions distributed with each new version

These were all relatively successful, but they can require a lot of work, and generally suffer from the ‘no one reads them anyway’ problem.

Solutions that I think are more effective:

  • Inline help – tool-tips, pop-ups, wizards. These are harder to ignore and give help at the time of need.
  • A No-Mistake workflow – Don’t allow invalid data or actions to occur. If you do, make sure that you…
  • Validate early – So users are told about errors in data before spending 12 hours rendering a scene.

So that’s my simple tools team test, how many does your team score?

  1. Do you have dedicated tools programmers?
  2. Do you deliver your tools atomically?
  3. Do you test tools releases?
  4. Can users submit bugs and feedback?
  5. Can your tools be used without a manual?

I’d love to include more questions, so which ones would you add?

All quiet on the CodeAmp front

Oops, haven’t updated recently? Bad Dave, no biscuit for you.

What’s going on?

Well as it happens I’m working very hard, just not on Rephrase. Rephrase is tantalisingly close to being a shippable product, but I was approached by a friend to work on something else for a little while. I’m working with a couple of other programmers and we’re building something big. It’s sooo much bigger than Rephrase could have been that I didn’t feel I could turn it down. So as it stands Rephrase is on pause until the New Year. By then hopefully we’ll have put together a prototype and we’ll be able to see how it looks. If it doesn’t look good, then I’ll come back to Rephrase full of new vigour and get it shipped.

Look a shiny thing!

If this sounds to you like I’m dropping everything to chase new shiny thing, then don’t worry, it feels a bit like that to me too. But it isn’t. There are several advantages to the new project, amongst them:

  • co-founders: I don’t think that co-founders are essential, however, I’ve already noticed what a difference it makes to my working day being able to have a chat over skype, talk about design decisions and see check-ins popping up in the Mercurial. It’s lovely.
  • It’s big: the new project has so much more scope than Rephrase it’s more in-line with how I want to take Code Amplifier but I would never have had the nerve and resources to aim so high. Rephrase was designed to be a product that I could manage on my own ( at least at the start ). What we’re doing now wouldn’t be possible on my own.

Onwards

So, apologies, if you urgently required an awesome tool to help with your localization woes, I’m afraid you’ll have to wait just a little bit longer. And apologies, I’m not going to be able to talk about what I’m doing again. It sucks, I know, I’d rather just be out with it, but it’s going to have to wait a little longer. Hopefully this will only last until early in the New Year.

The Rephrase work flow

After finally announcing Rephrase, I’ve been having some conversations about what it actually does and how it would benefit a game team. As usual with this sort of thing, I’ve had my head in this for long enough that the application seems obvious to me but to the rest of the world a few things need explaining. So I’ve knocked up a couple of diagrams that hopefully will explain why Rephrase is a good thing for the developer. Publishers and translators have different requirements in the localization process, but for developers the aim is to remove all the pain points that are associated with using generic tools ( or no tools at all ) to manage the large amount of data that localizing a modern game can generate.

Flow chart of localization work flow for a typical game

A typical work flow for localization of a game

Typical work flow

In the first diagram is what I think to be a example of a typical work flow when creating a game that is translated into multiple languages.

We start with the developer, who creates source data and adds it to the Master Spreadsheet. This is my major assumption, that most developers use a spreadsheet as their primary storage for localisation data ( at least text data ). This assumption seems to be borne out by the discussions that I’ve been having with developers.

Once the source data has been created someone divides up the master spreadsheet into separate sets suitable for each translator or agency. The translators do their work and the translations are entered into the new smaller spreadsheets and then returned to the developer. Someone back at the developer recombines the new translations into the master spreadsheet.

After the translations are made ( or as they are coming in ) an export process is run to convert the master spreadsheet into data that the game can use at run-time. Often this Export process is simply a VBAscript in the spreadsheet that converts the spreadsheet data into either a text or binary file which can be loaded by the game, or possibly it is converted into a source code file which is then compiled into the game directly.

Once the game is in test the real fun starts. If ( when ) the QA team find bugs in the localized data they submit these back to the development team who will flag the offending parts of the localized text as a bug ( and offer some description of what the bug is if it isn’t obvious ). Then, once a round of bug testing is complete the translation cycle begins again. This time not all of the assets need to be sent for translation, only those with bugs. Someone at the developer extracts the incorrect data from the master spreadsheet, along with description of the bug and then sends it off for translation. Once this data starts coming back in, the developer enters ‘Merge Hell’, multiple spreadsheets arrive, each with different languages and data, and must be merged into the master spreadsheet without introducing any new errors, or overwriting any previous bug fixes. Let’s hope, too, that none of the source data has been changed by the development team in the meantime. This is usually where we all start feeling sorry for the team member who is in charge of the localization.

Finally after many hours work, and lots of spreadsheet wrangling, all the bugs are fixed the data is re-exported one last time and the game ships. Hooray!

A flow chart showing the localization process using Rephrase

Work flow for localizing a game using Rephrase

Rephrase work flow

In the second diagram is the same work flow, only this time at the centre of the process is the Rephrase server. Hopefully it is clear from the diagram alone that this process is designed to be simpler and more straightforward, because Rephrase does a lot of the developer’s work for them.

We start with the developer again. They create the source data and add it through the web interface or via file import to the Rephrase server. Once the data is in the server and marked as ready to be translated it is made available to the translators next time they sign in to Rephrase. They take the source data and translate it adding the translations back to the server via the web interface or through a file import. Whilst the assets are marked as ready to translate, the developers can’t change the source so that we can be sure that when the data gets back from the translator, it is a translation of the correct source.

Exports can be made into a number of different formats, hopefully one that you can use directly in your game. If not, a post-export step can be applied to convert the data into the format you required.

The game is nearly finished now, and QA are battering it 24/7. Of course they find bugs in the localized content, but this time, instead of passing the bugs on to the development team to mark up, the QA department can simply re-open the flawed translation and mark it with a comment detailing what the bug is. Next time the translator signs in, the bug appears in their to-do list and gets fixed, with no intervention by the developers at all. If the QA team have the necessary skills and permissions, they could simply fix the issue themselves without even requiring the round-trip to the translator.

Of course all these actions are logged and browsable so the producers can see what activity is occurring and who is making changes.

Finally the last bug is fixed, the final export is made and the game is ready to ship. Hooray again!

Hopefully, this post makes it a little clearer how Rephrase can help game developers localize their games more easily and cut out on that Merge Hell.

Rephrase – taking the pain out of game localization

I’m pleased to announce the first product from Code Amplifier.

Rephrase logo

Rephrase – Localization data management

Rephrase is a server-based web application that provides change tracking and management for the localized assets in your game.

For publishers Rephrase means that localization can begin as soon as the developers add the assets to the server. Your localization and QA team can work on the assets while the game is being developed which saves you time as you localize in parallel to development. It also reduces any risk of slippage as the localization process is no longer tacked on to the end of development.

For developers Rephrase means you no longer have to worry about merging tens of Excel spreadsheets of localized data as it trickles in from many sources. You will always know if translations are out of date and need updating. Also Rephrase is designed specifically to be integrated into your tools pipeline. A comprehensive API means you can access and manipulate your data how ever you like.

For translators Rephrase means you can work from anywhere using the simple browser-based interface. Rephrase doesn’t want to get in your way so you can use your favourite translation tools and then add the results to Rephrase.

No set up required

Because Rephrase is a hosted web-application you will never have to worry about setting up and managing your server or ensuring that your users have the correct application installed on their local machines.

Sign up now

Rephrase can help you if you are an one-man indie outfit or a triple-A MMO game. We beginning our tests now so sign up and we’ll be in touch.

Rephrase HQ

Weekly Update: Code, code, business, code

Hi folks, very little to talk about this week, I’ve been toiling away on the front end of the web app with little chance to find anything interesting to talk about. To paraphrase the old saying “If you haven’t got anything interesting to say, don’t say anything at all”.

This week all I’ve got is to leave you with a rather large summary of the Business of Software conference that I read earlier in the week. Written by Patrick McKenzie, who runs the MicroISV on a shoe string blog, this post is full of stellar advice taken from the 3 days of talks at this years conference.

“Ideas that spread, win.”  Software is now about creating a tribe or otherwise achieving leadership of one.  Take AutoCAD, for example: it isn’t a product, it is a movement: we architects who use AutoCAD are here to change the industry away from using pencil&paper drafting.  You are either in the movement or opposed to it, because it is an existential threat to your business.

Seth Godin

Tell your sales representatives to call customers with low CHI [Customer Happiness Index] proactively and ask them what you can do to make them happy.  This simple technique saves 1/3 of canceled accounts, and is virtually free.

Dharmesh Shah

Quick tip when speaking to customers: “The most interesting people you will ever meet are the people who are most interested in you.”

-Paul Kenny

“The ineffective marketer asks you to buy too soon.”  A customer on their first visit could have multiple reasons why they are not currently prepared to buy your software:

Some words kill open rates if you use them in subject lines.  Reminder, Special, % off, and “Help [us]…” are all culprits.

Rob Walling

Oh, and once I’ve sorted out the landing page I’ll be finally be announcing the first Code Amplifier product next week sometime.

Weekly Update: User Experience and Developer Inspiration

Yikes, running a bit late on the update this week.

User Experience

This week I finally finished up the back-end work I was doing and started taking a proper look at the front end and user experience. I’ve been having the classic ‘blank page’ problem with the front end. There seems like so much to think about before you can start that you just never start. I’ve been trying to use various mockup tools to create some outlines of how the work flow will look, but I was finding it too difficult to do at my desk on the computer. Even with great tools like Balsamiq Mockups, looking at only one page at a time made it really difficult to visualise.

So I went old school, printed out 10 pages with a web browser picture on them and went downstairs to the dining room table with a pencil and a rubber.

Old School UX design

Old School UX design

So now I have 10 pages with the basic work flow laid out, something to work from when I’m coding up the html templates and a good visual image of how the application works.

I can see why tools like Balsamiq are good for collaboration with a distributed team, but for the first conceptual stages the tools you need are a big table, pencil, paper and an eraser.

Developer Inspiration

The reason I set up Code Amplifier in the first place is that I’m convinced that the games industry can make better games with better tools. I firmly believe that tools are by far the most important part of a game team. With shonky tools it doesn’t matter how cool your shader tech is, or that your AI can realistically out-flank a player over deformable geometry. Without good tools, your designers and artists are going to spend all their time updating xml files and waiting for the game assets to rebuild; or sitting around waiting because someone broke the build with a 2048×2048 texture; or entering localisation data into an Excel spreadsheet four times because you can’t merge a binary spreadsheet. They won’t have time to iterate their design and show off your cool flanking behaviour or tweak that camera fly-through so the awesome water shader gets seen up-close. And they’ll be miserable because they have to work late to get that localisation data in before the milestone tomorrow.

At Code Amplifier I want to get rid of (some of) those problems. So it was great to see Alex Evans ( no relation ) emphasizing a very similar point in his talk at BAFTA.

http://www.bafta.org/access-all-areas/videos/media-molecule,1383,BA.html

When you are working alone it’s easy to yo-yo between extreme confidence — “I’m going to sell thousands of licenses, it’s all going to be great” — and extreme pessimism — “Game developers don’t want tools made by someone else, and they’ll all have written their own, I’m doomed”. This video gave me a little boost into the confidence side. Someone else who thinks that tools are as important as I do.

There is hope yet.

1 for $5, 5 for $20

If it wasn’t enough that last week Eric Ries told me to stop wasting my time today I read a brilliant response on Hacker News to a developer who was depressed about the lack of attention for his brilliant ideas.

You seem to have been terribly misled. Only very rarely do products sell themselves. 99% of the time, the product is largely incidental to the sales process. Your idea doesn’t matter one jot, what matters is how well you can connect to customers and really sell to them.Let me tell you about a fine English gentleman by the name of Joe Ades, now sadly no longer with us. Joe wore Savile Row suits and lived in a three-bedroomed apartment on Park Avenue. He spent most nights at the Café Pierre with his wife, sharing a bottle of his usual – Veuve Clicquot champagne. You might assume that Joe was a banker or an executive, but in fact Joe sold potato peelers on the street for $5 each, four for $20.

I urge you, I implore you, I beg you, stop what you’re doing and watch Joe in action – http://www.youtube.com/watch?v=HCUct4NlxE0

That is what business looks like. Sometimes, once in a million, you luck upon a product so amazing the world beats a path to your door. For most of us, the best we can hope for is to be some chump with a thousand boxes of vegetable peelers. Anybody can sit out on the street with a box of peelers, but Joe sold them. Joe made his peelers sing, he made them seem like magic. He took a humble piece of stamped metal and created theatre. He did something so simple and strange and wonderful that people would buy a fistful of his peelers, just so they could tell their friends about this little Englishman they saw in Union Square.

Look at the Fortune 500, tell me what you see. I see grocery stores, drugstores, oil companies, banks, a funny little concern that sells sugar water. I see a whole lot of hard work and very few great ideas.

Forget about striking it big with a great idea, it’s just as childish and naïve as imagining that the tape you’re recording in your garage is going to make you a rockstar. Get out there and talk to customers. Find out what they need, what annoys them, what excites them. Build the roughest, ugliest piece of crap that you can possibly call a product. If you’re not ashamed of it, you’ve spent too long on it. Try and sell it. Some people will say “I’m not buying that piece of crap, it doesn’t even do X”. If X isn’t stupid, implement X. Some people, bizarrely, will say “yes, I will buy your piece of crap”. It is then and only then that you are actually developing a product. Until you’ve got a customer, it’s just an expensive hobby. Paying customer number one is what makes it a product.

The video of Joe Ades is amazing, I want to buy a potato peeler right now, but that last paragraph is the advice. Build it, get it out, talk to you customers and add the features that they need. Thanks jdietrich, whoever you are.

So, if you are reading this, if you’re a game developer, if your team are unhappy because localizing your game is wasting their time and their energy then drop me an email dave@codeampilfier.com and I can help you.

Thursday Update

Friday, already?

Wow, another week, that has gone really fast. Beautifully summed up by this tweet:

Sums it up nicely

http://twitter.com/#!/zssz/status/27435575863

I don’t have much to talk about this week as I’ve been head down and coding solidly for the last five days. I’ve embarked on a large re-factor of the code which I knew I’d have to do, but it’s been three days in now and all the automated tests are still failing. Half of me wants to ignore all the tests and push on just to get something out, but I know that I’m going to be heavily reliant on them once I’m launched. There simply isn’t going to be the time and money available for a large human QA department. So I’m putting the work in now, and sure it’ll save me time in the future. This sounds familiar, didn’t I say exactly the same thing last week? It’s going to turn into a mantra: “Do the work now, save time later”.

Lean Tech Hubbub

On Friday last week on the spur of the moment I popped back into London to visit the TechHub co-working space. Eric Ries of the Lean Startup Movement was in for a flying visit and he gave a good talk about the Lean Startup Basics. I’d heard most of his stuff before in various other talks, but still it’s nice to see him in person, and I hung around a bit to have a chat with a couple of friends who are doing various startup type things: Jos from HipSnip and Paulina from Inensu.

The ‘one thing’ that I took away from that talk though was a slide near the beginning that said:

Don’t waste peoples time

In other words, don’t spend a year building something that no one wants. These talks will almost never give you practical information that you can act on ( “Do X and you’ll succeed”, it just doesn’t work that way ), but sometimes there will be ‘one thing’ that sinks in and just bumps you onto a slightly different course.

In this case, that one slide has driven home the need to get my stuff out there into the public. Especially because in my case the people in that sentence is me and I’m not planning on wasting any of my time. I have a million ideas and if this one doesn’t work, then I want to get on to the next one as soon as possible.

I’m still a little way off from having the front end in a state that makes it worth showing to people, but the aim is to have a working version before the end of November.

Back to work then.

The Office Revealed

Philippa and I moved house a couple of weeks ago and part of the move was to make sure I could have a productive work environment, so the house we’ve picked was chosen in part for the home office environment. Here are a couple of pictures of the state of the office two weeks in…

Office Desk View

Office Desk View

My office is on the second floor in the converted attic. In this picture you can see:

The MacBook Pro – not getting much use at the moment apart from the odd Minecraft game

(Hidden behind desk) Ubuntu desktop – My main development machine

Monitor – Primary screen for the desktop, secondary for the Mac. It’s a Samsung SyncMaster, and I’ve slowly come to the conclusion that it’s a bit shit. I’ve found that light colours come out very light and it seems impossible to find a set up where I can see light greys/yellows and not have the whites look dimmed down. My second monitor won’t be one of these

Big ole’ PC desktop – I am looking after this for a friend who is in South Africa for a year. It hasn’t been booted yet, but I’m sure it’ll come in useful for Windows testing, if I can stop Philippa stealing it as a games machine.

MacMini – An old G4 MacMini which is running Ubuntu. This has no keyboard or monitor connected, I’m just ssh-ing in and using as a build server running BuildBot.

ReadyNAS NV+ – A four disk RAID server which is the media server for the house.

Extra router – Amazingly the wireless reaches the attic from downstairs, but most of the hardware here needs a wired connection, so I’ve got this spare router acting as a three-port switch. I’m going to have to get another one if I want to connect up the Big ole’ PC though as I’m already out of ports.

Aeron Chair – Absolute essential for spending 10 hours a day at the desk. This is another loan from my friend in South Africa.

Rubber Duck and Mod Nation Racer – Man Dolls left over from the SCEE days.

Cables – Everywhere.

And to show that the move is still work in progress here is the view in the other direction

Office Box View

Office Box View

Here you can see:

Boxes – Thousands of them

Sniper – The cat

Filing – That pile of paper on the floor. 🙂