Developer's Blog

The Dev Blog for the upcoming game Zero Sum Future, where our lead programmer walks through engine design, game design, and more!

Beta Test Post Mortem

That fresh blog post smell. Ah!

As our regular audience may or may not know, we recently conducted a beta event. How did it go? How did we intend for it to go? Did it go that way? Now that it's gone (but not really) how do we proceed in the coming days?

Get a cup of tea, it's quite a ride.

The Expectation:

The first public test event we ran was roughly 6 months ago and was almost exclusively friends and family (check out that blog post HERE). We had roughly 9-12 people (other than the Dev team) actually show up and give feedback, and it was a lot of the following:

  • The game crashes on start.

  • The game crashes when I place/delete a building.

  • The game crashes when you click the carrier UI element.

  • The game crashes when you try to get a multiplayer game going.

  • I have no idea what I'm doing. Can I build tanks?

I'm sure you can sense a theme here.

Going into the first real beta event, I was expecting a repeat performance: I was very confident that we'd get a large percentage of our testers complaining about stability. I was also worried on the performance front. On my machine, I can get a very comfortably get 60 fps with very reasonable settings, but my machine has a beefy GTX 980 in it. On an Intel integrated solution, we get about 10-15 fps, depending on the settings. So I was really expecting to get slammed for poor optimization, because, well, the engine is poorly optimized at the moment.

One thing we really didn't want was player confusion. We spent one month exclusively testing locally with whatever machines we had available to find everything we could to improve the user experience. We added tooltips, notifications, overhauled UI systems. Hell, we even shot a tutorial series and put it on Youtube (Which you can check out HERE)! I was very confident that people would watch them and become top-tier intergalactic moguls.

The Reality:

So, what was the feedback? Well, here's all of the complaints:

  • If I run the game, it crashes. No, I don't have OpenGL 4.3, why did you ask?

  • What the hell is the single-player AI doing? I'm so confused.

  • For some reason, the population hates my guts. Why does it do that?

  • My buildings stopped working. Help?

  • Why am I losing money?

  • It'd be really nice if I could auto-cast robotics (hint: you can)

  • I really wish that there were Tooltips and that they worked properly. Also there are spelling errors with the Tooltips.

  • The game is going REALLY fast, and I have no idea what's going on.

  • Single-player should really have a pause button.

  • The game is running poorly on my machine. Could it run less poorly?

Now, the praise:

  • The UI makes sense. You can understand what goes where and why.

  • The game looks very unique. We got a lot of this one.

  • I really like the idea of where you were going with it. I'll keep paying attention.

Yeah, this list is shorter. But that's not a bad thing, not really. It's always nice to know what to work on next.

How's the comparison to our expectations? Well, let's talk stability. The only real issue was with testers trying to run the game without the correct OpenGL support. This naturally crashes the game. Other than that, we had no issues! Except for this one graphical bug courtesy of Endrain over on our Discord:

Clearly not working as intended.

But other than THAT there were no stability issues. This was very surprising, but also very gratifying.

The biggest piece of negative feedback that I feel has to be addressed is the perceived complexity of the game. I was, for the longest time, worried about watering down the Gameplay elements of Zero Sum Future. Turns out, the game was actually very complicated and really, really difficult to understand. It doesn't matter how good the game is, if our players don't GET IT, then it's really for nothing.

We also need to worry about performance and optimization, though not nearly as much as I originally feared. A lot of feedback noted that while it didn't run great, it was acceptable for beta.

Overall, I'm fairly happy and confident about the end result. I'm also a little apprehensive, as we have a LOT of work to do in the coming weeks, but there is a fairly decent timetable for it.

Going Forward

I'm not going to mention the teeny tiny little bug fixes. Those are going to happen. Instead, I'm going to go over the major changes/additions that will merit their own, individual posts at a later date.

  1. We need a full single-player experience. The biggest complaint that we got this test was that the game was too difficult to understand, and thus we need a way to let players get comfortable before pitting them against one another. The creation of a single-player campaign will require writing, a script/map editor for our custom engine, and all the functionality that one would expect from those tools.

  2. I haven't made a post about it (yet), but we have a fairly powerful particle system built for Zero Sum Future. Right now, that system is not being utilized effectively. We're going to spend some time tinkering with it, and making sure things like the Sun and shuttle trails look up to par. I'll be composing another post about really powerful particle systems at a later date.

  3. The UI received some fairly mixed remarks. While most people thought it was fairly intuitive, the overall impression is that it doesn't convey enough information. We're going to add more UI elements – such as a resource budget menu, a full-blown unit management menu, and so on. We're also going to add floating icons to buildings so that you can tell what is happening to each one at a glance.

  4. Overall, very positive feedback about the sound design and the sound track. We're going to be expanding on that with more tracks and ambient sound effects. I'm still trying to get our sound designer to write a post explaining his process, so stay tuned.

  5. The overall optimization of the game needs some fixing. I can actually use some audience feedback on this one – if you know of a decent diagnosis/profiler tool for Nvidia/Intel chips and that answer isn't Nsight, hit me up.

  6. The multiplayer experience really needs a proper ranked ladder system. I'm a bit torn on this one, though – on the one hand, Valve doesn't want us talking about the Steamworks platform. On the other hand, the way that I'm going to deploy it is going to be impressively hacked together. I'll write a post on this at a later date, and explain how YOU can implement such a system yourself, regardless of platform.

All of the above (plus a massive number of small bug fixes and visual tweaks) are going to get crammed into a space of 3 weeks. We're going to be deploying a much-improved version to Steam on the 1st of October, and running a new beta event. It's our wish to run an open beta, but we might have to settle for passing out keys by hand again.

In the meantime, if you are interested in what you've read, what can you do? In the next couple of days, the keys we passed out for the event are going to start deactivating. If you'd like to keep playing, or if you haven't had a chance to play Zero Sum Future, drop us a line and we’ll see what we can do. Follow us on Social Media (Twitter, Facebook), Wishlist us on Steam, and keep an eye on this page.

As you can see, there's quite a bit of work to do, and as daunting as it is, I'm confident that we can get it done.