Project 4

Overview

The objective of this project is to gain experience building and training neural radiance fields (NeRFs) for novel view synthesis.

The goals for this project are as follows:

  • Implement a (tiny) Neural Radiance Field architecture for novel view synthesis.
  • Understand the characteristics of novel view synthesis and differentiable rendering.
  • Gain experience reimplementing network architectures by translating from text and figure descriptions to code implementations.

Instructions

  1. Download the project starter code
  2. Unzip the starter code and upload to Google Drive
    • Once unzipped, you should find a root directory titled ‘P4’. The ‘P4’ directory contains all starter code and files needed to complete this project. Please upload the ‘P4’ directory to your Google Drive.
  3. Open the *.ipynb and *.py files and implement features

    • We recommend implementing the features in a Google Colab environment. The Colab development environment can be accessed by double-clicking on each *.ipynb and *.py file within your Drive. Instructions for each feature are included in the pose_estimation.ipynb file.

    • We suggest starting by implementing the required features as they appear in the tiny_nerf_pytorch.ipynb notebook.

    • While working on the project, keep the following in mind:

      • The notebook and the python file have clearly marked blocks where you are expected to write code. Do not write or modify any code outside of these blocks.
      • Do not add or delete cells from the notebook. You may add new cells to perform scratch computations, but you should delete them before submitting your work.
      • Run all cells, and do not clear out the outputs, before submitting. You will only get credit for code that has been run.
      • To avoid experiencing Colab usage limits, save and close your notebooks once finished working.
  4. Submit your implementation for Autograder feedback
    • Once you have implemented a portion of the required features, you may submit your work for feedback from the Autograder. To receive feedback, download your *.ipynb and *.py files then upload them to the Project 4 Autograder. You may submit to the Autograder for feedback up to 2 times per day.
  5. Download final implementation
    • After implementing all features, save your work and download the completed *.ipynb and *.py files.
    • Your uploaded assignment should include tiny_nerf_pytorch.ipynb, tiny_nerf.py, pred_image.pt, and nerf.pth for this project.
  6. Submit your python and notebook files for grading
    • Upload your files to the Autograder for grading consideration. Your highest score will be used for final grades.

Deadline

This project is due on Sunday, March 31st at 11:59pm EST. We suggest starting as soon as possible.

Grading

This project will be graded by the Autograder. The project is worth a total of 20 points. You may submit to the Autograder for feedback up to 5 times per day.