Posted on March 10th, 2019

You will know more on below 3 things when you finish reading this blog.

Current challenges of cost and scalability and how Artificial Intelligence/Machine learning (AI) will help dealing with these challenges in 3D content creation?

  1. Current challenges of cost and scalability and how Artificial Intelligence/Machine learning (AI) will help dealing with these challenges in 3D content creation?
  2. Where do we stand in terms of using AI in our current production pipeline?
  3. Which technologies and algorithms we are using?

Today, ecommerce has been incorporating AI into everything such as for enhancing consumer experience, attracting users into accessing their platform and increasing conversion rates.

Whether you talk about Personalized Recommendations, where Amazon’s “item-to-item collaborative filtering” algorithm is providing tailor-made content direct to the homepage which resulted in an increase of its sales or you talk about voice commands where Voice-activated assistants like Amazon’s Echo, Google Home, and the Apple HomePod, or on mobile devices with Cortana and Siri, AI is ruling the world. Companies like Alibaba, Rakuten, eBay etc. are using Al for fake reviews detection, chatbots, product recommendations, managing big data, etc. Gartner predicts that 85% of interactions between customers and retailers will happen “without interacting with a human” by 2020.

While delivering a convenient browsing experience and a wider assortment ecommerce still had limitations. It could not overcome consumers’ desire to study the product from all possible angles prior to purchase, nor could it convincingly replicate the in-store experience. To overcome this limitation, 3D modelling is being a major contributor with which retailer can provide customers with a graphical 3D product representation. With this, a costumer can choose what part of the object they’d like to see, whether they want to zoom in or out the object or they want to rotate the object. All these features are accessible today with 3D modeling.
3 most common challenges for 3D models every eCommerce player is facing are –

  1. Scalability – Everyone wants to have 3D models created at earliest. We have also heard some clients wanting 1000 models or more in a month. On an average one model creation time including material is 10-12 hours so we are talking about 12,000 man hours of work which need 80 people in a month. Imagine how difficult it is to scale up so quickly and set-up a team that can deliver 1000 or more models.
  1. Cost – 3D is definitely an additional cost as its seen right now by decision makers. Executives have lot of pressure to reduce cost further down.
  1. Quality – Quality is important more than anything else, you want have realistic 3D models with right dimension and material.

SuperDNA realized this challenge earlier in the cycle and directly focused on AI experiments to integrate machine learning in creating 3D assets. We are investing into research and also creating our own AI/Machine Learning utilities which can help us to address these challenges. Below are some of the initiatives that we took –

  1. Repurposing – While working for large clients we are having separate production pipelines. We understand that after developing repetitive models and materials, we can use existing models and textures to bring the cost down. We have our own asset libraries as well which we have purchased which has more than 50,000 models and textures. We run our machine learning recommendation engine to provide best match to our 3D modeling and 3D texture artist so that they don’t need to start from scratch. Also preapproved models are already with high quality, so we face less quality issue.

Current Status of SuperDNA3D Lab – We started this in early 2018 and this utility is quite mature. We are able to save as much as 15% efforts/cost which we pass on to our customers.

Technology backgroundWe are using vggnet, a CNN trained using caffe deep learning framework. When you pass any image through the network it gives you pattern and shape features at the end of convolutional and fully connected (fc) layers, respectively. Those features are compressed using a compression matrix and then we use a nearest neighbor classifier to find the best match. The pattern and shape features of the database are precomputed and saved in csv files.

  1. Creation of textures/materials and 3D models.

3D texture creation

We also invested in scratch by creating low resolution textures directly to high resolution texture which means if we have scanned resolution textures then we create directly high resolution texture from our utilities. There are lot of tools available online but they miss out details and that’s what we are trying to address here.

3D model creation

We are in early stage of creating models from scratch for 3D directly from 2D images. We think if do breakthrough in this utility then we will be able to solve most of the problems. Human intervention will still be needed but it will a huge cost saving.

Technology Solution background –

SuperDNA is using an interesting approach to solve this problem. We came up with an idea of combining the researches done by people with the traditional approach of solving any computer vision problem. Utilizing the research from Wu and Hane’s papers, we created 2 neural networks, first one will be used to do the feature extraction and another network will predict the coordinate based on the features provided to this network. This approach is unique since we are combining 2 power networks, i.e., Convolutional Neural Network(CNN) for feature extraction and a recurrent neural network for predicting coordinates of the 3D object.

How we are doing it?

  1. We are trying to eat elephant bit by bit.
  2. We have very well integrated the AI utilities within our workflows so we test our own products and keep making them better every day.

In a nutshell –

To conclude, Integrating AI usage into 3D asset creation is a daunting task but results are positive. By working on re-purposing and creation separately we are able to address challenges of scalability, quality and costs.

Feel free to reach out to me if you want to know more … , Click here