Final Project

1051-753 Computational Photography

Final Project

Spring 2011

General Guideline

You have a good bit of freedom in choosing your final project. You are expected to implement one (or more) related research papers, or think of some interesting novel ideas and implement them using the techniques discussed in class. Below, we list some possible projects. However, you are welcome to come up with your own ideas - in fact we encourage this.

This project will require programming as well as the manipulation of cameras, light sources, images and/or video. Most likely, you will need to write a program with a user interface (i.e., a demo) so that you can show the results. It is also strongly recommended that you evaluate your system with varying input parameters. A comparion to other work is welcome, but not required. You can use your favoriate programming languages (e.g., Matlab, C, C++, Python, Java, etc.).

You are allowed to work on a project in groups of two. In your submission, please clearly identify the contribution of both group members. Please note that members in the same group will not necessarily get the same grade.

What to Submit?

The project is expected to be carried out through the remaining portion of the quarter. There will be three checkpoints: a project proposal, an intermediate milestone report, and a final project report.

Create a webpage for the final project. This page should include the three reports, the interesting implementation details, some results (e.g., images, videos, etc.), and so on. Your website should also includes downloadable source and executable code. Do not modify the website after the due date of the assignment. Also, send an email to jwgu@cis.rit.edu with the URL of your webpage BEFORE the due dates. Please also submit the proposal, the intermediate report, and the final report to dropbox in myCourse.

Project Proposal (Due: March 22, midnight) (5%)
This will be a short report (usually one or two pages will be enough). You will explain what problem you are trying to solve, why you want to solve it, and what are the possible steps to the solution. Please include a time table.
Intermediate Milestone Report (Due: April 19, midnight) (10%)
In this report, you will need to give a brief summary of current progress, including your current results, the difficulties that arise during the implementation, and how your proposal may have changed in light of current progress. Please keep in mind that the goal is to go around obstacles not stop working when you reach them.
Final Project Presentation(Due: May 10, midnight) (15%)
This will be a short presentation in class about your project. It will 15 to 25 minutes per project.
Final Project Report (Due: May 17, midnight) (10%)
Assemble all the materials you have finished for your project in the final report, including the motivation, the approach, your implemention, the results, and discussion of problems encountered. If you have comparison results or some interesting findings, put them on the webpage and discuss these as well.

Possible Project Ideas

1. Extension From Your Homeworks ...

In HW1, you will learn HDR imaging and tone mapping. It requires capturing an additional calibration image. Can you implement the algorithm in [Debevec et al. 1997], which calibrates the camera and generates HDR at the same time? There is another paper about camera radiometric calibration and HDR imaging. Check whether you can implement it.

In HW2, you will learn about Nokia N900 programming for some basic algorithm. Try to use this tool and implement more advanced image processing and computer vision algorithms with the N900 platform, such as flutter shutter photography, flash/no flash photography, depth from focus/defocus, image stitching, face detection and tracking, etc.

In HW3, you will learn about photometric stereo in the simplest setting: known light sources, Lambertian surfaces. What if we do not know the light source directions? What if the surfaces are non-Lambertian? Here is a paper about photometric stereo under unknown light sources. Here is a paper of photometric stereo for general materials. Implement one or both features in photometric stereo.

In HW4, you will learn how to separate direct and global illumination using active illumination. Active illumination has been used in many computer vision tasks for scene recovery, such as photometric stereo, structured light range scanning. Choose one of the scene recovery application and implement it with and without direct/global separation. Compare the scene recovery results and discuss the differences.

2. Light Field Viewer (Thanks to Prof. Szymon Rusinkiewicz and Prof. Jason Lawrence)

You will implement a light field viewer based on the techniques described by Levoy & Hanrahan 96. Given the input as a light field, the viewer you need to implement should allow the user interactively move the camera, and generate an image corresponding to that particular view. In other words, it should perform the following operations:

For each pixel, generate a ray starting at the camera position and in the pixel's direction
Intersect the ray with the uv and st planes, finding the closest samples in each
If both intersections are valid, look up the color for that pixel, else color the pixel black.
Once all pixels have been assigned a color, use glDrawPixels() to render to the screen

The basic operation performed by the light field viewer is looking up the color corresponding to a given ray in space. Since the light field is represented by "slabs" (pairs of planes in space), you will need to first derive code for intersecting a ray in space with a rectangular region of a plane (specified with the region's four corners). If you have not taken a graphics course, perhaps you should skip this.

Here are some single-slab light field data (buddha, crayons, dragon, lion). Here is the format of the light field data. Here is multi-slab light field data.

3. Scene Clustering Based on Its Time-Varying Appearance Profiles

Many times when we observe real world scenes, as the light changes, from the time-varying appearance, we can easily group certain regions together, according to either their surface normals or their material properties. Here is an interesting paper about surface normal clustering using a moving light source (no calibration is needed). here is a paper about scene clustering and factorization of time-lapsed videos. Choose one and implement the algorithm.

4. Smart Image/Video Resizing: Seam Carving

This project is to re-implement the Seam Carving paper by Avidan & Shamir 07, a cool way to resize images and videos. There is a follow-up paper that extends the idea for videos.

Write code to compute the edge-based measure for an input image.
Implement the dynamic programming code to find the optimal seam to remove along one direction.
Implement the algorithm to deal with resizing along both x and y directions (i.e., to find the optimal seams-order)
Implement other applications discussed in the paper.
Discussions: is there a better measure? How does this algorithm compare with other simple resizing algorithms, e.g., to remove an optimal column instead of an optimal seam? What are the limitations of this algorithm?
(Not easy) You might want to extend this to videos, as shown in their follow-up paper.

5. Link with your own research topics

Can some of the ideas in computational photography be useful for your own research, e.g., using coded sensors, cameras, or light sources? It would be great if you can think of a project that is related to your current research. Discuss with me if you are not sure about the topic.