Pointclouds and PCL Types

Introduction Caesar._PCL

A wide ranging and well used point cloud library exists called PCL which is implemented in C++. To get access to many of those features and bridge the Caesar.jl suite of packages, the base PCL.PointCloud types have been implemented in Julia and reside under Caesar._PCL. The main types of interest:

These types are conditionally loaded once Colors.jl is included (see Requires.jl),

using Colors, Caesar
using StaticArrays

# one point
x,y,z,intens = 1f0,0,0,1
pt = Caesar._PCL.PointXYZ(;data=SA[x,y,z,intens])

# etc.
Caesar._PCL.PointCloudType
struct PointCloud{T<:Caesar._PCL.PointT, P, R}

Convert a PCLPointCloud2 binary data blob into a Caesar._PCL.PointCloud{T} object using a field_map::Caesar._PCL.MsgFieldMap.

Use PointCloud(::Caesar._PCL.PCLPointCloud2) directly or create you own MsgFieldMap:

field_map = Caesar._PCL.createMapping(msg.fields, field_map)

Notes

  • Tested on Radar data with height z=constant for all points – i.e. 2D sweeping scan where .height=1.

DevNotes

  • TODO .PCLPointCloud2 convert not tested on regular 3D data from structured light or lidar yet, but current implementation should be close (or already working).

References

  • https://pointclouds.org/documentation/classpcl11pointcloud.html
  • (seems older) https://docs.ros.org/en/hydro/api/pcl/html/conversions8hsource.html#l00123

Conversion with ROS.PointCloud2

Strong integration between PCL and ROS predominantly through the message types

  • @rosimport std_msgs.msg: Header, @rosimport sensor_msgs.msg: PointField, @rosimport sensor_msgs.msg: PointCloud2.

These have been integrated through conversions to equivalent Julian types already listed above. ROS conversions requires RobotOS.jl be loaded, see page on using ROS Direct.

Caesar._PCL.PCLPointCloud2Type
struct PCLPointCloud2

Immutable point cloud type. Immutable for performance, computations are more frequent and intensive than anticipated frequency of constructing new clouds.

References:

  • https://pointclouds.org/documentation/structpcl11pclpoint_cloud2.html
  • https://pointclouds.org/documentation/classpcl11pointcloud.html
  • https://pointclouds.org/documentation/common2include2pcl2point__cloud8h_source.html

See also: Caesar._PCL.toROSPointCloud2

Caesar._PCL.PointXYZType
struct PointXYZ{C<:Colorant, T<:Number} <: Caesar._PCL.PointT

Immutable PointXYZ with color information. E.g. PointXYZ{RGB}, PointXYZ{Gray}, etc.

Aliases

  • PointXYZRGB
  • PointXYZRGBA

See

  • https://pointclouds.org/documentation/structpcl11pointxyz.html
  • https://pointclouds.org/documentation/point__types8hppsource.html
Caesar._PCL.HeaderType
struct Header

Immutable Header.

See https://pointclouds.org/documentation/structpcl11pclheader.html

Missing docstring.

Missing docstring for Caesar._PCL.PointField. Check Documenter's build log for details.

Missing docstring.

Missing docstring for Caesar._PCL.FieldMapper. Check Documenter's build log for details.

Aligning Point Clouds

Caesar.jl is currently growing support for two related point cloud alignment methods, namely:

ScatterAlign for Pose2 and Pose3

These factors use minimum mean distance embeddings to cost the alignment between pointclouds and supports various other interesting function alignment cases. These functions require Images.jl, see page Using Images.

Caesar.ScatterAlignType
ScatterAlign{P,H1,H2} where  {H1 <: Union{<:ManifoldKernelDensity, <:HeatmapGridDensity}, 
                              H2 <: Union{<:ManifoldKernelDensity, <:HeatmapGridDensity}}

Alignment factor between point cloud populations, using a continuous density function cost: ApproxManifoldProducts.mmd. Supports very large density clouds, with sample_count subsampling for individual alignments.

Keyword Options:

  • sample_count::Int = 100, number of subsamples to use during each alignment in getSample.
  • bw::Real, the bandwidth to use for mmd distance
  • rescale::Real
  • N::Int
  • cvt::Function, convert function for image when using HeatmapGridDensity.
  • useStashing::Bool = false, to switch serialization strategy to using Stashing.
  • dataEntry_cloud1::AbstractString = "", blob identifier used with stashing.
  • dataEntry_cloud2::AbstractString = "", blob identifier used with stashing.
  • dataStoreHint::AbstractString = ""

Example

arp2 = ScatterAlignPose2(img1, img2, 2) # e.g. 2 meters/pixel 

Notes

  • Supports two belief "clouds" as either
  • Stanard cvt argument is lambda function to convert incoming images to user convention of image axes,
    • Geography map default cvt flips image rows so that Pose2 +xy-axes corresponds to img[-x,+y]
      • i.e. rows down is "North" and columns across from top left corner is "East".
  • Use rescale to resize the incoming images for lower resolution (faster) correlations
  • Both images passed to the construct must have the same type some matrix of type T.
  • Experimental support for Stashing based serialization.

DevNotes:

  • TODO Upgrade to use other information during alignment process, e.g. point normals for Pose3.

See also: ScatterAlignPose2, ScatterAlignPose3, overlayScanMatcher, Caesar._PCL.alignICP_Simple.

Note

Future work may include ScatterAlignPose2z, please open issues at Caesar.jl if this is of interest.

Iterative Closest Point

Ongoing work is integrating ICP into a factor similar to ScatterAlign.

Caesar._PCL.alignICP_SimpleFunction
alignICP_Simple(X_fix, X_mov; correspondences, neighbors, min_planarity, max_overlap_distance, min_change, max_iterations, verbose, H)

Align two point clouds using ICP (with normals).

Example:

using Downloads, DelimitedFiles
using Colors, Caesar

# get some test data (~50mb download)
lidar1_url = "https://github.com/JuliaRobotics/CaesarTestData.jl/raw/main/data/lidar/simpleICP/terrestrial_lidar1.xyz"
lidar2_url = "https://github.com/JuliaRobotics/CaesarTestData.jl/raw/main/data/lidar/simpleICP/terrestrial_lidar2.xyz"
io1 = PipeBuffer()
io2 = PipeBuffer()
Downloads.download(lidar1_url, io1)
Downloads.download(lidar2_url, io2)

X_fix = readdlm(io1)
X_mov = readdlm(io2)

H, HX_mov, stat = Caesar._PCL.alignICP_Simple(X_fix, X_mov; verbose=true)

Notes

  • Mostly consolidated with Caesar._PCL types.
  • Internally uses Caesar._PCL._ICP_PointCloud which was created to help facilite consolidation of code:
    • Modified from www.github.com/pglira/simpleICP (July 2022).
  • See here for a brief example on Visualizing Point Clouds.

DevNotes

  • TODO switch rigid transfrom to Caesar._PCL.apply along with performance considerations, instead of current transform!.

See also: PointCloud

Visualizing Point Clouds

See work in progress on alng with example code on the page 3D Visualization.