On-Manifold Operations

Caesar.jl and libraries have adopted JuliaManifolds/Manifolds.jl as foundation for developing the algebraic operations used.

The Community has been developing high quality documentation for Manifolds.jl, and we encourage the interested reader to learn and use everything available there.

Separate Manifold Beliefs Page

See building a Manifold Kernel Density or for more information.

Why Manifolds.jl

There is much to be said about how and why Manifolds.jl is the right decision for building a next-gen factor graph solver. We believe the future will show that mathematicians are way ahead of the curve, and that adopting a manifold approach will pretty much be the only way to develop the required mathematical operations in Caesar.jl for the forseable future.

Are Manifolds Difficult? No.

Do you need a math degree to be able to use Manifolds.jl? No you don't since Caesar.jl and related packages have already packaged many of the common functions and factors you need to get going.

This page is meant to open the door for readers to learn more about how things work under the hood, and empower the Community to rapidly develop upon existing work. This page is also intended to show that the Caesar.jl related packages are being developed with strong focus on consolidation, single definition functionality, and serious cross discipline considerations.

If you are looking for rapid help or more expertise on a particular issue, consider reaching out by opening Issues or connecting to the ongoing chats in the Slack Channel.

What Are Manifolds

If you are a newcomer to the term Manifold and want to learn more, fear not even though your first search results might be somewhat disorienting.

The rest of this page is meant to introduce the basics, and point you to handy resources. Caesar.jl and NavAbility support open Community and are upstreaming improvements to Manifolds.jl, including code updates and documentation improvements.

'One Page' Summary of Manifolds

Imagine you have a sheet of paper and you draw with a pencil a short line segment on the page. Now draw a second line segment from the end of the first. That should be pretty easy on a flat surface, right?

When the piece of paper is lying flat on the table, you have a line in the Euclidean(2) manifold, and you can easily assign [x,y] coordinates to describe these lines or vectors. Note coordinates here is a precise technical term.

If you roll the paper into a cyclinder... well now you have line segments on a cylindrical manifold. The question is, how to conduct mathematical operations concisely and consistently indepependent of the shape of your manifold? And, how to 'unroll' the paper for simple computations on a locally flat surface.

How far can the math go before there just isn't a good recipe for writing down generic operations? Turns out a few smart people have been working to solve this and the keyword here is Manifold.

If you are drinking some coffee right now, then you are moving the cup in Euclidean(3) space, that is you assume the motion is in flat coordinates [x;y;z]. A more technical way to say that is that the Euclidean manifold has zero curvature.

What if you are concerned with the orientation of the cup too–-as in not spill the hot contents everywhere–-then you might actually want to work on the SpecialEuclidean(3) manifold – that is 3 degrees of translational freedom, and 3 degrees of rotational freedom. You might have heard of Lie Groups and Lie Algebras, well that is exactly it, Lie Groups are a special set of Group Manifolds and associated operations that are already supported by JuliaManifolds/Manifolds.jl.

Things are a little easier for a robot traveling around on a flat 2D surface. If your robot is moving around with coordinates $[x,y,\theta]$, well then you are working with the coordinates of the SpecialEuclidean(2) manifold. There is more to say on how the coordinates $[x,y,\theta]$ get converted into the $\mathfrak{se}(2)$ Lie algebra, and that gets converted into a Lie Group element – i.e. $([x;y], \mathrm{RotMat}(\theta))$. More on that later.

Perhaps you are interested in relativistic effects where time as the fourth dimension is of interest, well then the Minkowski space provides Group and Manifold constructs for that – actually Minkowski falls under the supported Lorentz Manifolds.

The point here is that the math for drawing line segments in each of these manifolds above is almost exactly the same, thanks to the abstractions that have already been developed. And, many more powerful constructs exist which will become more apparent as you continue to work with Manifolds.

7 Things to know First

As a robotics, navigation, or control person who wants to get started, you need to know what the following terms mean:

• Q1) What are manifold points, tangent vectors, and user coordinates,
• Q2) What does the logarithm map of a manifold do,
• Q3) What does the exponential map of a manifold do,
• Q4) What do the vee and hat operations do,
• Q5) What is the difference between Riemannian or Group manifolds,
• Q6) Is a retraction the same as the exponential map,
• Q7) Is a projection the same as a logarithm map,

Know it sounds like a lot, but the point of this paragraph is that if you are able to answer these seven questions for yourself, then you will be empowered to venture into the math of manifolds much more easily. And, everything will begin to make sense. A lot of sense, to the point that you might agree with our assesment that JuliaManifolds/Manifolds.jl is the right direction for the future.

Although you will be able to find many answers for these seven questions in many places, our answers are listed at the bottom of this page.

Manifold Tutorials

The rest of this page is devoted to showing you how to use the math, write your own code to do new things beyond what Caesar.jl can already do. If you are willing to share any contributions, please do so by opening pull requests against the related repos.

Using Manifolds in Factors

The best way to show this is just dive straight into a factor that actually uses a Manifolds mechanization, and RoME.Pose2Pose2 is a fairly straight forward example. This factor gets used for rigid transforms on a 2D plane, with coordinates $[x,y,\theta]$ as alluded to above.

A Tutorial on Rotations

A Tutorial on 2D Rigid Transforms

Existing Manifolds

The most popular Manifolds used in Caesar.jl related packages are:

Group Manifolds

• TranslationGroup(N) (future work will relax to Euclidean(N)).
• SpecialOrthogonal(N).
• SpecialEuclidean(N).
• _CircleEuclid LEGACY, TODO.
• AMP.SE2_E2 LEGACY, TODO.

Riemannian Manifolds (Work in progress)

• Sphere(N) WORK IN PROGRESS.

Creating a new Manifold

JuliaManifolds.jl is designed to make it as easy as possible to define your own manifold and then get all the benefits of the Manifolds.jl ecosystem. Follow the documentation there to make your own manifold, which can then readily be used with all the features of both JuliaManifolds as well as the Caesar.jl related packages.

Answers to 7 Questions

Q1) What are Point, Tangents, Coordinates

A manifold $\mathcal{M}$ is a collection of points that together create the given space. Points are like round sprinkles on the donut. The representation of points will vary from manifold to manifold. Sometimes it is even possible to have different representations for the same point on a manifold. These are usually denoted as $p$.

Tangent vectors (we prefer tangents for clarity) is a vector $X$ that emanates from a point on a manifold tangential to the manifold curvature. A vector lives in the tangent space of the manifold, a locally flat region around a point $X\in T_p \mathcal{M}$. On the donut, imagine a rod-shaped sprinkle stuck along the tangent of the surface at a particular point $p$. The tangent space is the collection of all possible tangents at $p$.

Coordinates are a user defined property that uses the Euclidean nature of the tangent space at point $p$ to operate as a regular linear space. Coordinates are just a list of the indepedent coordinate dimensions of the tangent space values collected together. Read this part carefully, as it can easily be confused with a conventional tangent vector in a regular Euclidean space.

For example, a tangent vector to the Euclidean(2) manifold, at the origin point $(0,0)$ is what you likely are familiar with from school as a "vector" (not the coordinates, although that happens to be the same thing in the trivial case). For Euclidean space, a vector from point $p$ of length $[x,y]$ looks like the line segment between points $p$ and $q$ on the underlying manifold.

This trivial overlapping of "vectors" in the Euclidean Manifold, and in a tangent space around $p$, and coordinates for that tangent space, are no longer trivial when the manifold has curvature.

Q2) What is the Logarithm map

The logarithm X = logmap(M,p,q) computes, based at point $p$, the tangent vector $X$ on the tangent plane $T_p\mathcal{M}$ from $p$. In other words, image a string following the curve of a manifold from $p$ to $q$, pick up that string from $q$ while holding $p$ firm, until the string is flat against the tangent space emminating from $p$. The logarithm is the opposite of the exponential map.

Multiple logmap interpretations exist, for example in the case of $SpecialEuclidean(N)$ there are multiple definitions for $\oplus$ and $\ominus$, see [2.15]. When using a library, it is worth testing how logmap and expmap are computed (away from the identity element for Groups).

Q3) What is the Exponential map

The exponential map does the opposite of the logarithm. Imagine a tangent vector $X$ emanating from point $p$. The length and direction of $X$ can be wrapped onto the curvature of the manifold to form a line on the manifold surface.

Q4) What does vee/hat do

vee is an operation that converts a tangent vector representation into a coordinate representation. For example Lie algebra elements are tangent vector elements, so vee([0 -w; w 0]) = w. And visa versa for hat(w) = [0 -w; w 0], which goes from coordinates to tangent vectors.

Q5) What Riemannian vs. Group Manifolds

Groups are mathematical structures which often fit well inside the manifold way of working. For example in robotics, Lie Groups are popular under SpecialEuclidean(N) <: AbstractGroupManifold. Groups also have a well defined action. Most prominently for our usage, groups are sets of points for which there exists an identity point. Riemannian manifolds are more general than Lie groups, specifically Riemannian manifolds do not have an identity point.

An easy example is that the Euclidean(N) manifold does not have an identity element, since what we know as $[0,0]$ is actually a coordinate base point for the local tangent space, and which just happens to look the same as the underlying Euclidean(N) manifold. The TranslationGroup(N) exists as an additional structure over the Euclidean(N) space which has a defined identity element as well as a defined operations on points.

Q6) Retraction vs. Exp map

Retractions are numerically efficient approximations to convert a tangent vector into a point on the manifold. The exponential map is the theoretically precise retraction, but may well be computationally expensive beyond the need for most applications.

Q7) Projection vs. Log map

The term projection can be somewhat ambiguous between references. In Manifolds.jl, projections either project a point in the embedding to a point on the manifold, or a vector from the embedding onto a tangent space at a certain point.

Confusion, can easily happen between cases where there is no ambient space around a particular manifold. Then the term projection may be moot.

In Manifolds.jl, an inverse retraction is an approximate logmap of a point up from the manifold onto a tangent space – i.e. not a projection. It is important not to confuse a point on the manifold as a point in the ambient space, when thinking about the term projection.

It is best to make sure you know which one is being used in any particular situation.