Playtesting Lost Ruins of Arnak: Expedition Leaders (Part 2)

In this part, we’ll talk a bit more about what information we were actually tracking and looking into, and why digital playtest is not everything and real-world tables actually matter.

If you’ve missed Part 1 of this series, you can check it out here.

PROCESSING (AND EVALUATING) THE DATA

During playtesting we were tracking a lot of data, both from the feedback forms the players filled out and from the game stats. The data we’ve gathered was highly rewarding, but processing and evaluation of all this data were also quite challenging.

As was mentioned before, this data is invaluable to us – the data and the option to replay each game session directly on BGA are both the strongest tools we have to make sure different elements of the gameplay are balanced. The expansion is highly asymmetrical, so we had to be extra careful with tinkering and tuning.

One of the most important things we tracked was the average and maximum scores of each leader, to see how consistent their performance is. From this data, we saw which characters needed tweaking, and over time we were also able to define which leaders were more suited for beginner players and which are more difficult to play.

We also tracked other things like what scores were achieved on which of the two new research tracks, the number of cards gained and played, the number of turns per leader and per round, how often different leader-specific bonuses were used in case of the Falconer and the Mystic, and much more.

Often, when interpreting the data, we even had to go through particular gameplays to identify what caused some significant score deviations. Sometimes these anomalies were indeed caused by that leader’s abilities, but often there were other factors in play as well, like big differences in players’ experience or an unusual combination of conditions (like cards in the card row, sites discovered, etc.) that were more favorable for that particular game situation.

Sometimes, the differences between beginner and more advanced players were also quite significant – it was visible both in the base game and in the expansion implementations that some cards and strategies were often neglected by the beginners but turned out to be quite powerful in the hands of experienced players. We had to take these differences into account as well before we could start drawing conclusions.

DIGITAL AND PHYSICAL WORLDS COLLIDE

Of course, digital playtesting is not everything, and it was crucial for us to playtest Expedition Leaders with people on the “real-world” table. Some things that are working smoothly when everything is automated might turn out to be not ideal when translated to a physical environment. To identify this, we had to see how people were operating the game with their hands. How is the tablespace working? Is anything too fiddly? Are players forgetting anything? These and more questions were constantly asked and observed when we brought the expansion to the physical table – various live playtesting events, limited of course by the current COVID situation, helped us check and tweak the experience.

The Mystic, for example, had a special token that went through some iterations after seeing players handling it on the table. Under specific circumstances, players were supposed to store the Fear cards under this token. However, it turned out to be too fiddly and the token often ended up buried under the cards instead of being on top of the pile. So we reworked our original idea and the token became a board on which you could store the cards.

This is just one of the many changes we made thanks to the feedback we got from the people attending our testing events. We feel very fortunate that so many amazing people were willing to help us with the playtesting! Many things were fine-tuned and perfected thanks to their help and we believe these changes, though sometimes seemingly subtle, made a world of difference. 


Thank you for joining us on this journey! Come back next week on Thursday, September 23, to read the third and last part of this series.

Read Part 1

Playtesting Lost Ruins of Arnak: Expedition Leaders (Part 1)

Today, after a month of wild playing, experimenting, and iteration,  the online playtesting of Expedition Leaders is over. Now seemed like a good time for us to look back and reflect on this challenging yet so interesting and rewarding process that we’ve been engaged in for the past few weeks.

Our report is split into 3 parts – we’ll be releasing parts 2 and 3 on a weekly basis, so you can expect the next two articles to pop out on September 16 and 23. We hope you’ll enjoy this little peek behind the curtain and let’s begin!

INTO THE UNKNOWN

As usual, the prototype has been developed and discussed in-house for quite some time, with many CGE colleagues playing and commenting on the expansion. And once we saw that the core was solid and we were happy with how the expansion worked and how playing it felt, we decided to take one step forward and let other players from outside of the company join the process. In general, this playtest was important to us on multiple levels.

The base game of Lost Ruins of Arnak is played and enjoyed by many people – the game has a strong fan base with very invested and committed players. This motivated us, even more, to make the expansion as polished as possible, because people enjoy it so much and we don’t want to disappoint our fellow board gamers (which is not a small pressure in itself).

The second biggest reason for the playtest being so important is the element of asymmetry that the expansion brings to the table. Each new leader has their own features, abilities, starting decks, and their play styles are just different. Asymmetry in games is great when it works, as it keeps the games more exciting and fresh, but it’s also so incredibly tough to balance.

Playtesting launched on 10 August, and it was one of the biggest and most intense playtest events we’ve ever conducted.

Its scale definitely added to the feeling of uniqueness and gratitude that we’ve felt throughout the process. Almost 100 active players from all over the world joined our efforts and more than 560 games were played throughout the span of one month – this number roughly translates to an average of 18 games per day which is an incredible number. In the final week of the testing, there was a point where at least one game was played at any hour of the day, including workdays (and nights). It was exciting to see players so engaged!

GETTING READY

To make this playtest happen, we’ve had to do a lot of preparation and work on our side before the action even began.

The most obvious step was to put together a concise rules overview because up until that point, the rules were in our heads and in multiple docs and sheets, and we’ve had to translate all this into rough and functional bullet points that people who’ve never seen the expansion would understand.

We’ve also had to prepare a Discord server and, after its virtual gates were open, manage the community there. In order to populate this server, we had to make sure we picked enough players and, of course, prepare the digital implementation itself – the virtual table where the most important things would happen.

Preparing the digital version of the expansion was made much easier because we already have a quality implementation of the base game on BoardGameArena, so the groundwork was already laid with lots of basic rules and processes being in place. However, there was still a lot of work left to be done for the expansion to work. 

This process was made even easier for us because the creator of the base game BGA implementation, Adam Španěl, joined CGE and was in much closer contact with the team and deeply involved in the design process itself. Adam managed to prepare the expansion’s implementation much faster than we could’ve hoped for. During playtesting itself, he was also quick to help the community by fixing bugs and implementing design changes in record time. It was almost unbelievable how fast the implementation progressed in Adam’s hands 🙂

BoardGameArena also served as a strong source of playtesters, since there are many active and engaged Lost Ruins of Arnak players on the platform. That player community has a great knowledge of the game and is well-versed in different strategies and play styles suited for different game situations. We picked some of the best, most experienced, and most active players and offered them a chance to help us perfect the expansion content. Besides the BGA player base, we also talked to many folks in our Czech community and invited them to join us on our endeavor. In the end, we’ve managed to bring together 100 players for whose feedback, comments, and ideas we are very, very grateful. You guys have put a lot of time and effort into this – thank you!


This is all from us for this week’s post – come back next week, on September 16, to check out part 2 of this series 🙂