Farm3D

Farm3D is a tool developed by the University of Oxford that creates articulated 3D animal models from a single image. It uses a unique approach by leveraging a 2D diffusion-based image generator, like Stable Diffusion, to train a network that reconstructs detailed 3D shapes without needing real 3D data. This method allows Farm3D to generate 3D models with fine details such as legs and ears, even though it was not trained on real images.

The tool supports controllable 3D synthesis, enabling users to adjust lighting, swap textures between models of the same animal category, and animate the shapes. It works with both real images and images generated by Stable Diffusion, producing 3D assets in seconds. Farm3D factors input images into components like articulated shape, appearance, viewpoint, and light direction, giving users control over the final model.

Farm3D also introduces the Animodel dataset, a collection of textured 3D meshes of articulated animals such as horses, cows, and sheep, with realistic poses. This dataset serves as a benchmark to evaluate the quality of single-view 3D reconstruction for articulated animals.

This tool is ideal for researchers, digital artists, and developers interested in 3D animal modeling, animation, and synthesis without requiring extensive 3D training data. Its novel use of diffusion models for virtual supervision and scoring sets it apart from traditional 3D reconstruction methods.

Farm3D's approach reduces the need for costly 3D annotations by using synthetic views generated and critiqued by diffusion models during training. This results in a monocular reconstruction network that is fast, flexible, and capable of producing high-quality 3D animal models with controllable features.