The story goes something like this: You want to run playtesting at scale, so you give a build to the testers, they play for hours, and the feedback that comes back is either too thin (maybe a Discord message such as “it was fun”), or too late (a Google Form filled out three days after the session, with the details already fuzzy).
Using this feedback to figure out what players actually feel when playing, and when they are feeling it while playing, is easier said than done. But System Era solved this problem by implementing the feedback surveys directly in-game using FirstLook’s SDK. This is what the flow looks like.
In FirstLook you can easily set up a custom homepage with your own branding, domain, and messaging.
System Era set up a branded homepage at starseeker.firstlook.gg where players could sign up to participate in playtests in 2 easy steps:
Link their Discord account
Answer which platform they primarily play on
As a company, you have the option to customize the sign-up flow however you like. You can automatically let anyone applying in, or you can have a waitlist where you select the players best suited for you.
Simple as that! Players who signed up were put on a waitlist, and System Era could easily go through the applications, accepting the players they saw fit.
01 · Branded landing page02 · Link Discord03 · Choose platform04 · On the waitlist
”We benefited enormously from using FirstLook as our playtesting platform leading up to STARSEEKER’s launch. The immediate ease of use, the rapidly expanding capabilities, and the highly communicative and engaged team at Pragma make this an easy recommendation for developers looking to streamline and maximize their pre- and post-launch testing.”
The FirstLook SDK lets a studio trigger surveys from inside the running game, served either through FirstLook’s pre-built UI that studios can easily customize to match their own branding, or the Steam Overlay, so players never have to alt-tab, leave the session, or remember to come back later. The studio decides when a survey fires (after a tutorial, for example, or after a Steam-auth event). Because of this, players can respond while the experience is still fresh.
The team at System Era ran several targeted in-game surveys:
Tutorial Opinion, a focused opinion-scale check on how helpful the new tutorial actually felt
End of Expedition #3, general sentiment after a player’s third expedition
End of Expedition #6, the same check-in after six
End of Expedition #12, again after twelve, to see how players felt as they got deeper into the loop
Narrative Survey, a survey to test story content, comprehension, and UI implementation
For context: in STARSEEKER, 1–4 players go on Expeditions, a timed adventure where you explore alien planets and gather as much loot as possible before oxygen runs out.
This approach allowed the team to watch players’ sentiment evolve the more they played, both in regards to the gameplay but also the story, how the game fit into the Astroneer universe, and UI implementation, rather than getting one flat snapshot at the end of the test.
”The FirstLook team is very responsive and open to any kind of feedback. After each of our External Tests we were able to gather a large amount of player feedback to be analyzed thanks to their tools. A lot of the pain points we had with the tool were quickly added to the roadmap or were already on the way to be addressed by our next test.”
Doing an in-game survey doesn’t just give “more responses.” It gives better responses, tied to known context.
Because the SDK runs alongside FirstLook’s Steam integration (Steam playtime tracking, Steam-auth-as-survey-trigger, expedition counts, tutorial completion), every response comes with the player’s actual progression attached. The team doesn’t have to ask “how far did you get?” because they already know. They can ask the questions that matter.
A few things that fall out of that:
Cohort-level analysis is automatic. Filter responses by ET1, ET2, ET3, by platform, by playtime bucket, by tutorial completion. Compare ET1’s tutorial sentiment to ET3’s after the team rewrote it.
Sentiment curves over time. The 3 / 6 / 12 expedition cadence shows whether players are warming up to the gameplay loop or burning out, and exactly when the inflection happens.
No “please remember and report back” tax. Players answer in the moment, with the experience still loaded in their heads.
”FirstLook helped us manage players from sign-up to playtest while making it easy to communicate with them and gather feedback the entire time. The FirstLook team is also very supportive, both by readily helping us troubleshoot and by taking our suggestions and turning them into features!”
Marc Lawless, Sr. Player Support Representative, STARSEEKER
Lastly, we asked System Era’s COO, Veronica Peshterianu and team a few questions about playtesting and their experience with FirstLook and how it benefited them, and they generously gave us the time of day.
Q
Before FirstLook, how were you collecting playtest feedback, and what was actually failing about it? Did a piece of feedback ever come in too late or too vague to act on?
A · Veronica Peshterianu, COO, System Era
Before FirstLook, we only held internal playtests and we were using tools like Retrium and Miro to gather feedback. That is a reasonable solution for an internal team of a few dozen where you can follow up on any confusing piece of feedback as easily as sending a Slack DM. But that process absolutely does not scale at hundreds and thousands of players. FirstLook helped us manage the much larger scale of incoming feedback without having to rely on manual collection and analysis. So in other words, we sought a solution before we hit a failure state to avoid that outcome from the get go.
Q
If you’d had to build in-game surveys yourselves, what would that have realistically cost you in engineering time?
A · Veronica Peshterianu, COO, System Era
The key benefit of FirstLook to us was the unified, integrated environment that was easy to get started with and successfully merged onboarding, key distribution, surveys, analysis, and sentiment monitoring under one roof instead of having to use five different external tools.
Q
Did anything the surveys surfaced change a decision you made about the game? The tutorial rewrite between ET1 and ET3 is mentioned. What did the data actually tell you, and what did you change because of it?
A · Veronica Peshterianu, COO, System Era
We made several adjustments to our Tutorial based on the player data we received from FirstLook, primarily around time spent on certain activities by players. Our original Tutorial was far longer, with more traversal sections up front followed by a hub-and-spoke model of completing objectives. Players clearly signaled that they felt well educated on traversal much sooner than the segment ended, and didn’t enjoy covering the same ground going back and forth between the Tutorial NPC and objectives. In the updated version, we highly streamlined the movement and interaction basics section, and then directed the player through a single loop around the space with automatically updating objectives.
Q
The 3 / 6 / 12 expedition cadence was built to watch sentiment evolve. Did you see a moment where players warmed up or started to drop off? Did the curve match what you expected?
A · Veronica Peshterianu, COO, System Era
We saw a clear progression where players were still figuring things out at Expedition 3, and a steady progression to fuller understanding and therefore fuller fun. By Expedition 12, players were more engaged, more confident in their choices, and finding more long term fun in STARSEEKER.
If you want to run your own playtest program on FirstLook, get in touch.
