CS4277/CS5477 Lab 3-Relative pose estimation with 8-point algo- rithm Solved

1 CS4277/CS5477 Lab 3-1: Relative pose estimation with 8-point algo- rithm

 In [2]: import cv2
         import matplotlib.pyplot as plt

         import numpy as np
         import h5py
         import scipy.io as sio
         from eight_point import compute_fundamental, compute_essential, decompose_e
         %matplotlib inline

1.0.1 Introduction

In this assignment, you will get to estimate the essential and fundamental matrix by using eight point algorithm. As discussed in the lecture, images taken from different views should fulfill the epipolar constraint, which can be used to estimate the fundamental and essential matrix. You will first estimate the fundamental and essential matrix with 15 correspondences provided in the dataset. Then you will decompose the essential matrix to find rotation and translation between two views. The decomposition will give 4 feasible camera poses and you will select the the correct pose by chriality check.

This assignment is worth 10% of the final grade.
References: * Lecture 6
Optional references: * Richard I. Hartley. In Defence of the 8-point Algorithm

1.0.2 Instructions

This workbook provides the instructions for the assignment, and facilitates the running of your code and visualization of the results. For each part of the assignment, you are required to complete the implementations of certain functions in the accompanying python file (eight_point.py).

To facilitate implementation and grading, all your work is to be done in that file, and you only have to submit the .py file.

Please note the following:
1. Fill in your name, email, and NUSNET ID at the top of the python file. 2. The parts you need to implement are clearly marked with the following:

```
""" YOUR CODE STARTS HERE """

""" YOUR CODE ENDS HERE """
```

, and you should write your code in between the above two lines.

3. Note that for each part, there may certain functions that are prohibited to be used. It is important NOT to use those prohibited functions (or other functions with similar func- tionality). If you are unsure whether a particular function is allowed, feel free to ask any of the TAs.

1

1.2 Part 2: Estimate Fundamental Matrix from Point Correspondences

In this part, you will implement the 8-point algorithm to estimate the fundamental matrix. For any pair of matching points xi ↔ x′i in two images, the 3 × 3 fundamental matrix is defined by the equation:

x′TFx = 0
Let f be the 9-vector made up of the entries of F in row-major order, we get:

(x′x, x′y, x′, y′x, y′y, y′, x, y, 1)f = 0
From a set of n point matches, we obtain a set of linear equations of the form:

Af = 0

The solution for f is the singlar vector corresponding to the smallest singular value of A. Then you will enforce the singularity constraint to F matrix such that the rank of F is 2. Note that the normalization step is very important here to for accurate estimation.

You can verify your estimation by visualizing the epipolar lines in both images, where the epiploar lines will pass through all matching points. The helper function plot_epipolar_line() is provided for visualization

Implement the following function(s): cv2.findFundamentalMat() * Prohibited Functions: cv2.findFundamentalMat()
* You may use the following functions: np.linalg.svd()

3

1.3 Part 3: Estimate Essential Matrix from Point Correspondences

In this part, you will also implement the 8-point algorithm to estimate the essential matrix. The steps are the same with the fundamental matrix estimation except for that :

1. The normalization step: For each correspondence xi ↔ x′i, compute K−1xi ↔ K′−1x′i. K and K′ are the camera calibration matrices which are given in the intrinsics.h5 file. Note we only give one camera calibration matrix here because the two images are taken by the same camera.

2. The singlarity constraint: The essential matrix should have two similar singular values, and third is zero.

Implement the following function(s): cv2.findEssentialMat()
* Prohibited Functions: cv2.findEssentialMat()
* You may use the following functions: np.linalg.svd(), np.linalg.inv()
Note that the your estimated essential matrix may be different from the results estimated by

using cv2.findEssentialMat(), because the cv2.findEssentialMat() use a different algorithm.

1.4 Part 4: Two-view Relative Pose Estimation
In this part, you will extract the relative rotaion R and translation t from the essential matrix E

accordint to:

E = ⌊t⌋×R.

The essentrial matrix can be decomposed into 4 feasible camera poses, and you will select the correct one by cheriality check. Specifically, the 3D structure can be computed with the linear tri- angulation method, and the 3D points should appear in front of both cameras. Note that we assum that the rotation and translation of the first camera are identity matrix and zeros respectively.

4

Implement the following function(s): cv2.recoverPose() * Prohibited Functions: cv2.recoverPose()
* You may use the following functions: np.linalg.svd()

In [5]: trans = decompose_e(E, K, data1, data2)

5