Point Cloud Segmentation

What would be objects that appear in the pcd but are not obstacles? For the most part, any free space on the road is not an obstacle, and if the road is flat it’s fairly straightforward to pick out road points from non-road points. To do this we will use a method called Planar Segmentation which uses the RANSAC (random sample consensus) algorithm.

• Segmentation: is the association of certain points to objects.

Point Cloud Processor

It has function for filtering, segmentation, clustering, load/save PCD, ...

Segmenting the Plane with PCL

We use max iteration and distance tolerance. We create a list of inliers (which is a list of ints) which is all the indecies from that point cloud that belong to the plane. We separate the points that belong to the road from the points that belong to the scene.

• Segmentation uses an iterative process. More iterations have a chance of returning better results but take longer.

• The segmentation algorithm fits a plane to the points and uses the distance tolerance to decide which points belong to that plane. A larger tolerance includes more points in the plane.

Timer

This can be useful for measuring how long it takes to run the function. If it’s taking a really long time to process the segmentation, then the function is not going to be useful running in real-time on a self-driving car.

// Time segmentation process
auto startTime = std::chrono::steady_clock::now();

// TODO:: Fill in the function to segment cloud into two parts, the drivable plane and obstaclesauto endTime = std::chrono::steady_clock::now();
auto elapsedTime = std::chrono::duration_cast<std::chrono::milliseconds>(endTime - startTime);
std::cout << "plane segmentation took " << elapsedTime.count() << " milliseconds" << std::endl;

Inliers

• Inliers (a vector of ints) are indices for the fitted plane. Using these inliers we can create the plane point cloud and obstacle point cloud.

• We create 2 point clouds; the road and the obstacle point clouds. The road points are the same as the inliers; because the inliers represent all the indices of points that belong to the plane, and that plane is the road. The plane cloud has member variable points, which is a vector of <PointT>

• All the indices from our point cloud that belong to the plane (road) are pushed to the plane cloud. We can use pcl::extract to create the obstacle point cloud from the inliers.

std::pair<typename pcl::PointCloud<PointT>::Ptr, typename pcl::PointCloud<PointT>::Ptr> segResult(obstCloud,planeCloud);

Obstacle Cloud

• In order to generate the obstacle cloud, all the points that are not inliers are kept.

• Inliers are removed from the reference cloud by using the negative method.

• Note: The above example is using 100 iterations, and a distance tolerance of 0.2 meters. You are highly encouraged to play around and experiment with these values! This point cloud is very simple and 100 iterations are way more than needed. You can also monitor how changing the iterations affects the time it takes for the segmentation function to process using the functions predefined timer log. Before rendering the two clouds you will want to remember to turn off rendering from the input cloud done in the previous lesson, otherwise the clouds will all overlap, and it will be hard to distinguish the segmented ones. The renderPointCloud function includes color options (Red,Green,Blue). By default the cloud is white if no color is specified. Here the obstacle cloud as is rendered as red, and the plane cloud as green.

RANSAC Algorithm

RANSAC stands for Random Sample Consensus, and is a method for detecting outliers in data. RANSAC runs for a max number of iterations, and returns the model with the best fit. Each iteration randomly picks a sub-sample of the data and fits a model through it, such as a line or a plane. Then the iteration with the highest number of inliers or the lowest noise is used as the best model.

• Iterative Method

• Each iteration randomly picks subset of the points

• Fit a model to the points

• Iteration with the most inliers to the model is best

• Hyper parameters are: No. of iterations, and distance tolerance.

The situation is: →We have a data and we know that there is a line in it. But the data also has a ton of outliers. Applying linear regression here and taking the average of all points does not make sense, since it will also take the average of the outliers and will result in a very bad line that does not represent the actual line that you have.

• One type of RANSAC version selects the smallest possible subset of points to fit. For a line, that would be two points, and for a plane three points. Then the number of inliers are counted, by iterating through every remaining point and calculating its distance to the model. The points that are within a certain distance to the model are counted as inliers. The iteration that has the highest number of inliers is then the best model.

• Other methods of RANSAC could sample some percentage of the model points, for example 20% of the total points, and then fit a line to that. Then the error of that line is calculated, and the iteration with the lowest error is the best model. This method might have some advantages since not every point at each iteration needs to be considered. It’s good to experiment with different approaches and time results to see what works best.

• The max iteration size to run depends on the ratio of inliers to the total number of points. The more inliers our data contains the higher the probability of selecting inliers to fit the line to, and the fewer iterations you need to get a high probability of selecting a good model.

Segmentation of Ground Plane from a Point Cloud

We do that because we want to know the height above ground for the different objects.

From sensor fusion perceptive, the ground plane is the road, and you get reflections from lane markings. So you can have all these lane markings in the ground plane as separate objects.

Do hills affect Lidar?

They do affect int in terms of FOV, you don't see over the crown of a hill, until you are relatively far up. There are ways to detect the slope of a hill using lidar.

So in general if doesn't affect the lidar, it is just sth in the environment that you would detect.