Scene crafter: 3D scene generation for game assets

Abstract

The creation of high-quality 3D scenes for game assets is traditionally a time-intensive and costly process, requiring significant manual effort and expertise. This project explores an innovative approach to automate and streamline 3D scene generation by integrating Score Distillation Sampling (SDS) into SceneDreamer, a state-of-the-art framework for unbounded 3D scene generation from 2D image collections. While SceneDreamer demonstrates impressive capabilities, its outputs are limited in flexibility and diversity. By leveraging SDS, which utilizes pre-trained diffusion models to guide 3D generation without explicit 3D data, this work aims to enhance the variety and customizability of generated scenes while reducing development time and cost. The proposed integration of SDS into the SceneDreamer pipeline explores the potential of text-guided 2D priors to generate diverse and adaptable 3D scenes. Initial results, though not yet perfect, demonstrate the promise of this approach in achieving more flexible and diverse 3D scene generation. This work contributes to the growing field of automated 3D content creation, offering a pathway to more efficient and scalable asset production for games and other applications.