Update 01/18/2023: Added comment from Roblox.

Concerned about what her children might be seeing in ultra-popular and controversial game creation platform Roblox, one parent decided to take a look for herself. She spent six hours playing the games intended for 5 year-olds, before deleting her kids’ accounts without delay.

Carolyn Velociraptor, the Twitter handle of one games industry veteran who’s worked at Bethesda, Ubisoft, and BioWare, says the idea began out of a curiosity about the darker aspects of monetising kids and how service games target such a young audience, but she didn’t expect to find what she did: “weird pedo stuff and bathroom voyeur games and suicidal idealization”.

Roblox has various age ratings which affect which accounts can see a given piece of content, with the idea being that the permissions on a child’s Roblox account are set by a parent or responsible adult. The content creators fill in a questionnaire to apply these ratings to whatever they’ve made. Thus, even though the criteria are set by Roblox Corp, it is not age rating player-made experiences manually, which is simultaneously something of a cop-out and an innate problem for a platform of this size. Few big tech companies will come out and say it, but manual pre-moderation at this scale is not something they want to invest in (or consider practical), and thus the ‘solution’ is to rely on the community and moderate based on reports. 

The most disturbing theme that Carolyn encounters is a sub-genre of bathroom games.

“You use the toilet,” writes Carolyn. “Your character’s bottom clothing item often disappears. The door stays open, and other players watch you. The toilet turns yellow/brown”.

Speaking as a parent, young children being fascinated by toilets and bodily functions is unsurprising. But one of the fuzziest areas when it comes to moderation is roleplay experiences, and unfortunately that’s what these bathroom games seem to really be about.

“In more than a few of these games, there’s an area that forces your character into a reclined position (by “slipping on water” or getting knocked by other players),” writes Carolyn. “And in some cases, you glitch/get stuck there”.

When this happens, and Carolyn offers two videos showing it, players begin moving their avatars in a back-and-forth motion at the bottom of the recumbent avatar’s legs. It seems like a clear attempt to represent sex between two avatars, and one that multiple players repeat across the videos.

Carolyn picks out other examples of this “creep factor”, including a bathroom game in which users can trap each other in stalls and take photographs. And the bathroom games are just one strand of this problem: Survive Pepa [sic] Piggy is apparently for all ages, with a nightmarish header image of a bloodied Daddy Pig having been stabbed in the gut (Peppa Pig survival games are their own entire genre on Roblox).

The bathroom games are a perfect example of why children and user-generated content can be such a bad mix, and the boundaries can get so blurred. In theory, would I have a problem with one of my kids playing a daft videogame where the character could pee and poo? Nope. But as soon as you see the examples of what’s going on in the Roblox bathroom games, it is clear this is nightmare fuel and not something children should be seeing.

“I’ve been wondering why Roblox would suddenly approve thousands of games (especially NSFW-ish games like RP worlds) for kids age 8 & under,” writes Carolyn, and references some of the reporting on Roblox Corp’s exploitation of developers on the platform. “As more parents started using the age-gated restriction settings, it creates a problem: fewer active players engaged with the platform’s highest-grossing experiences (which are impossible to actually make safe for kids). Apparently they just don’t care anymore”.

I contacted Carolyn Velociraptor on Twitter, who since her observations went viral has been dealing with some pushback from elements of the Roblox fandom.

“One thing I do have to clarify from my thread (after speaking with several Roblox developers) is that the way Roblox enacted parental controls has changed,” writes Carolyn, “several months ago, restricted accounts were only allowed a list of games manually selected by Roblox (mostly tycoons, building simulators, not many RP experiences). This all changed when Roblox decided to put the responsibility on developers, having them fill out a survey to determine their game’s age rating, and then using those ratings for the new parental control system.

“Some developers seem to have felt pressured to answer the survey in a way that would earn the all-ages rating, since age-gating locks out significant portions of the Roblox user base”.

So it’s inarguably in developers’ interests to try and avoid those restrictions, purely because if you don’t get that “all ages” rating then your audience is severely limited. The bigger concern is the change itself, which seems to suggest that Roblox has more-or-less (in Carolyn’s words) “intentionally offloaded some responsibility”.

That is a bigger takeaway than the bathrooms.

There is a devil’s advocate position for Roblox that is all about the sheer scale of the platform, and the seemingly insurmountable nature of the problem because of that. But scale goes both ways, and Roblox Corp makes an enormous amount of money (the company’s revenue in December 22 was between $430-$439 million, with 61.5 million daily users, and its shares are up 30% year-on-year).

If you’re making that much money from an audience that can skew so young, then questions of responsibility and ethical business have to be answered. Roblox Corp doesn’t tend to publicly engage with criticism, preferring instead to issue investor updates, keep the interviews to a minimum, and count up the dough. With regulators looking suspiciously at how big tech companies protect children, and the UK even threatening to jail bosses who don’t do it well enough, that posture may become untenable.

Update 01/18/2023: Roblox provided us with a lengthy statement, parts of which are excerpted here:

“In September last year, we launched Experience Guidelines, which are designed to provide upfront information about what an experience contains and provide an age recommendation. Experience Guidelines centre around the content of the experience created by the developers: and there is an important distinction to be made here between experience content, and user behaviour.

“User behaviour within experiences (which is what was highlighted in that thread) is dynamic and always changing so the user behaviour that may occur within an experience is not rated within Experience Guidelines.”

The experience guidelines are Roblox’s age ratings.

“To determine the suitability of content for each age category, we examined global industry standards and consulted child development experts to guide our policies. The experiences highlighted in the Twitter thread are marked as “All Ages” because the content is generally suitable for all ages and may contain infrequent mild violence and/or light unrealistic blood”.