Epic Games Metahuman Creator – Art Imitating Life With Arvind Neelakantan

Anyone with experience in 3D Modeling would know that making people is hard. What’s worse, if your game called for a crowd shot, that’s a whole crowd of people you’d need designed, modeled and animated. We spoke to Arvind Neelakantan, Technology Evangelist at Epic Games for India and South East Asia about Epic Games’ newest tool, Metahuman Creator. While it’s not going to completely eliminate the time and effort of producing 3D people, it definitely cuts down on the process greatly.

Unsurprisingly, the uses for easily produced high-quality humans are near infinite. According to Arvind, the team has high hopes for Metahuman Creator, though they aren’t bottlenecking it to any single use.

“Metahumans opens the door to have a lot of realistic looking characters. I do see a lot of applications in the gaming space. In fact there were also different kinds of games and also a sense of what games would also look like in the future. Which is where indie games really shine because they’re usually in the forefront or the cutting edge of what games can really do”, he says.

While the assets can definitely be used for games, he’s hoping the readily-accessible human models means we’ll expand our idea of what we can use games for.

“Typically games will really entertain us, they’ll create a social space where they can connect with other people. But there are also other kinds of games where we learn through these digital humans. Such as we can explore the anatomy for example, how we can better improve our sports, or even how to improve our dancing skills if you’re like me I definitely want to do that. We can also have a game where we can choose our wardrobe and maybe even buy it in the future”, he says.

“Some of these are more immediate and obvious, like using them as NPCs and characters in your game. Some of these can even be speculative and futuristic but one thing is certain for indie developers. Digital humans will be one of the fundamental building blocks of many many technologies that’s going to be coming in the future”, he adds.

While the sky’s the limit, Arvind didn’t agree with some tech doomsayers who think the future is 100% automation. According to Arvind, computers can’t replace real performances, and Metahuman Creator would be more of a great support for a great performance.

“I don’t believe that we’ll ever replace actors or artists with this technology. What we want to do is we want to automate this time consuming and repeatable processes which are typically very mechanical and repeatable. Keep an eye on what the fundamental art skill is vs what a temporary production requirement that needs to be filled is. For any developer you just need to keep a good balance of these two”, he says.

All Sorts Of People

So, the next-biggest question when it comes to the Unreal Engine Metahumans is taking them to task for just how diverse they can really be. Games like Cyberpunk 2077 famously flaunted a “robust” character creator, but at the end of the day you were still either a Dorito or a sentient letter S, albeit with varied genital preferences. With Metahuman Creator though, Arvind says that representation is key, and that they can only go further with it in future.

“One of the top priorities for us is to build upon the database that we have to make sure you have a diverse set of appearances and characters out of this tool. I cant really say that we’re 100% there because we are not as diverse as we can be. But we are getting there and we’re trying to improve as we continue the tool’s capabilities”, Arvind says. While they’re not the be-all-end-all of representation yet, he says there’s definitely plans to improve the tools users will have in the future, too. 

“We’re also working to further improve the fidelity of the deformation rig and to add the workflows once users receive the assets itself and will do our best to provide creative folks and artists and developers around the world with produces diverse and high fidelity digital humans at a fraction of time and cost which is what Metahumans is really about”. 

One such way they were looking into improving representation is making people with visible disabilities. Currently, the Metahuman Creator isn’t able to do things like prosthetics. While anyone with knowledge of 3D software might be able to fix that, the aim of Metahumans is to make this level of potential accessible to other people, so that means making visible disabilities would be added to the future plans list for Metahumans, too.

“We should also talk about people with disabilities because that’s important. At this moment, it cannot have asymmetrical characters because there’s a large number of variations that disabilities can produce. What we are working on definitely is looking into this and making sure we have sufficient level of quality that we can add to the tools that can then make this additional advancements to the data collection tools to add this kind of representation for people who have asymmetrical body types or disabilities”, Arvind says.

Getting Around The Problem With Making Real People In Metahuman Creator

One huge problem with making real people that any creative should be familiar with is the concept of the Uncanny Valley. Basically, there’s a point as you approach photorealism where things start to look unnatural. The best example of this would be any of the virtual celebrities posthumously appearing in movies, which while always praised at the time always seem to look off not a year later.

“The uncanny valley is pretty notorious, right?”, Arvind laughs. “Even the best looking characters that people have painstakingly created still has that. For starters creating digital humans is a painstaking process that typically requires a high level of expertise and a lot of time. So each facial feature in Metahumans have been refined individually down to the finest detail.”. He also noted while getting static images to look good, there’s also another layer added into making it look natural in motion.

“All the rigging and animated characters adds this entire level of complexity because you might have seen certain instances that a photograph looks a lot real but as soon as the digital human moves, you get the uncanny valley. And we want to erase the stress of starting from scratch and toll through all these technical hurdles that come from making these great looking characters and of course Metahuman Creator is a realization of decades of work and we continue improving upon it but rest assured, we are adding tools that make sure, at least that we kind of close down on the uncanny valley as much as possible”.

One way getting out of the Uncanny Valley, especially when dealing with something with performance capture like Metahumans does, is getting the details down. Arvind says with the Metahumans Creator, users will definitely have access to plenty of these.

” So you might have seen the Metahumans creator videos along with the sample characters we released you can play around with, you can download it for free from our marketplace. There are a lot of attributes that you can quickly fine-tune that also include stuff like skin tone, eye color , facial feature, hairstyles, textures and down to even subtle details like peach fuzz, fine wrinkles that you get around your nose or eyes, and whatnot. You can really go down to extremely minute details like these. You can also add things like your spots that you can see on the skin and whatnot. It gets pretty minute”, he says.

Of course, where does one get the idea to have that many minute details? According to Arvind, the answer is real life. They did this with the help of studios like 3Lateral and Cubic Motion, both of whom were acquired by Epic Games as part of the development for MetaHumans. Arvind explained how the work of the two studios were quintessential in the development of MetaHumans.

“Just to wind back a few years ago, we bought the experts from companies like 3Lateral and Cubic Motion and made them as part of the Epic family. And the mission for bringing them into Epic was, to make this technology available to the world, and to have this Metahuman tool, to bring this tool out into the world and for everyone to see it. So the technology behind this, its a combination of 3Lateral’s 3D and 4D scanning technologies to digital double pipeline, which is a technology that was built using cubic motion’s animation technology. And there’ll also be a special projects team that have been working internally in Epic Games for a lot of years”, he says.

He went on to explain the process of making the large databases behind MetaHumans, which involves scanning real people and then making sure the data they get works within the confines of the engine.

“The process is a very carefully supervised process. It starts from first scans of real life people to the last QA checks of the health of the database. So it starts off with our custom built scanners and that goes through different teams that manage data and presentation, fitting, cleanup, shape modeling, rigging texturing, look dev and database integration, all of that happening with this tool. Each database variant then goes through many tests which also includes privacy tests for different aspects like functionality, fidelity and user experience, At the same time we’re also working on automating a lot of these components of this process.”, he says.

Sensing we were getting a little lost in the details, Arvind graciously broke down the workflow process behind making Metahuman Creator into a bite-sized chunk:

“It’s scanners. It’s 3D scanners that can do this for you. I’m kind of breaking it down into extremely simplistic terms, fundamentally its 3D scanners which thanks to 3Laterals and Cubic Motion’s technologies along the internal body”, he laughs. 

With all this talk about highly detailed models, you might be worried that the final product could only be run on a top-of-the-line machine. Considering the aim is to make assets for games on everything from mobile to high-end PCs, Arvind says that you won’t have to worry about the prospect of adding a model and breaking your whole game.

“So they currently can run smoothly on a PC and console but we are also working right now to optimize them for lower end devices like mobile. That also includes stuff like AR apps and such. As we mentioned right now, hair simulation and clothes simulation, they are pretty computationally expensive, and we are working to make sure that you are able to optimize if not the simulation at least the human being itself will run well on any device”, he says.

Metahuman Creator is a completely free addition to the Unreal Engine, acting as another tool to help you make high quality games for free. According to Arvind the tool won’t be beholden to any kind of licensing- totally free high-quality asset creation at your fingertips. There’s even a free sample you can test out, with two fully-formed Metahumans that you can even puppet using the LiveLink app on iOS devices.

Thanks again to Arvind for talking to us about this exciting new piece of technology with Metahuman Creator, and Epic Games for giving us the opportunity to do so.

Join Us!