Abstract
We consider the problem of editing 3D objects and scenes based on open-endedlanguage instructions. The established paradigm to solve this problem is to usea 2D image generator or editor to guide the 3D editing process. However, thisis often slow as it requires do update a computationally expensive 3Drepresentations such as a neural radiance field, and to do so by usingcontradictory guidance from a 2D model which is inherently not multi-viewconsistent. We thus introduce the Direct Gaussian Editor (DGE), a method thataddresses these issues in two ways. First, we modify a given high-quality imageeditor like InstructPix2Pix to be multi-view consistent. We do so by utilizinga training-free approach which integrates cues from the underlying 3D geometryof the scene. Second, given a multi-view consistent edited sequence of imagesof the object, we directly and efficiently optimize the 3D objectrepresentation, which is based on 3D Gaussian Splatting. Because it does notrequire to apply edits incrementally and iteratively, DGE is significantly moreefficient than existing approaches, and comes with other perks such as allowingselective editing of parts of the scene.