DistributedFactorGraphs Functions

Filter nodes in a DFG based on a predicate function. This function modifies the input nodes vector in place, removing nodes that do not satisfy the predicate.

• For cross-backend compatibility: Use only the standard predicates (==, <, <=, >, >=, in, contains, startswith, endswith) when you need your code to work with both in-memory and database-backed DFGs. These are likely to be supported by database query languages and are defined in std_numeric_predicates and std_string_predicates.

• For in-memory only operations: You can use any Julia predicate, since you have full access to the data and Julia's capabilities. This is more flexible but will not work if you later switch to a database backend or another programming language.

Standard predicates

• Numeric predicates: ==, <, <=, >, >=, in
• String predicates: ==, contains, startswith, endswith, in

getBiadjacencyMatrix(dfg; solvable)

Get a matrix indicating adjacency between variables and factors. Returned as a named tuple: B::SparseMatrixCSC{Int}, varLabels::Vector{Symbol) facLabels::Vector{Symbol). Rows are the factors, columns are the variables, with the corresponding labels in varLabels,facLabels.

Notes

• Returns ::NamedTuple{(:B, :varLabels, :facLabels), Tuple{SparseMatrixCSC, Vector{Symbol}, Vector{Symbol}}}

getGraphLabel(dfg)

getSolvable(dfg, sym)

Get 'solvable' parameter for either a variable or factor.

getSolvable(node)

Variables or factors may or may not be 'solvable', depending on a user definition. Useful for ensuring atomic transactions.

Related:

• isSolveInProgress

getSolverParams(dfg)

getSummaryGraph(dfg)

Get a summary graph (first-class citizens of variables and factors) with the same structure as the original graph.

Notes

• this is a copy of the original.
• Returns ::GraphsDFG{NoSolverParams, VariableSummary, FactorSummary}

getTypeDFGFactors(_)

Function to get the type of the factors in the DFG.

getTypeDFGVariables(_)

Function to get the type of the variables in the DFG.

isSolvable(node)

Variables or factors may or may not be 'solvable', depending on a user definition. returns true if getSolvable > 0 Related:

• getSolvable(@ref)

rebuildFactorCache!(dfg, factor)
rebuildFactorCache!(dfg, factor, neighbors)

Method must be overloaded by the user for Serialization to work.

setSolvable!(node, solvable)

Set the solvable parameter for either a variable or factor.

Default non-parametric graph solution.

Standard parametric graph solution (Experimental).

sortDFG(vars; by, kwargs...)

Convenience wrapper for Base.sort. Sort variable (factor) lists in a meaningful way (by timestamp, label, etc), for example [:april;:x1_3;:x1_6;] Defaults to sorting by timestamp for variables and factors and using natural_lt for Symbols. See Base.sort for more detail.

Notes

• Not fool proof, but does better than native sort.

Example

sortDFG(ls(dfg)) sortDFG(ls(dfg), by=getLabel, lt=natural_lt)

Related

ls, lsf

toDot(dfg)

Produces a dot-format of the graph for visualization.

Notes

• Returns ::String

toDotFile(dfg)
toDotFile(dfg, fileName)

Produces a dot file of the graph for visualization. Download XDot to see the data

Note

• Default location "/tmp/dfg.dot" – MIGHT BE REMOVED
• Can be viewed with the xdot system application.
• Based on graphviz.org

listNeighborsSecondary(dfg, label)

List the second order neighbors of a given node.

ls(dfg, node; solvable)

Retrieve a list of labels of the immediate neighbors around a given variable or factor.

ls(dfg; whereSolvable, whereTags, whereType, whereLabel)

List the DFGVariables in the DFG. Optionally specify a label regular expression to retrieves a subset of the variables. Tags is a list of any tags that a node must have (at least one match).

Notes:

• Returns Vector{Symbol}

lsTypes(dfg)

Return Vector{DataType} of all unique variable types in factor graph.

lsTypesDict(dfg)

Return ::Dict{DataType, Vector{Symbol}} of all unique variable types with labels in a factor graph.

lsf(dfg; whereSolvable, whereTags, whereType, whereLabel)

List the DFGFactors in the DFG. Optionally specify a label regular expression to retrieves a subset of the factors.

Notes

• Return Vector{Symbol}

lsf(dfg, _)

Lists the factors of a specific type in the factor graph. Example, list all the Point2Point2 factors in the factor graph dfg: lsf(dfg, Point2Point2)

Notes

• Return Vector{Symbol}

lsfPriors(dfg)

Return vector of prior factor symbol labels in factor graph dfg.

Notes:

• Returns Vector{Symbol}

lsfTypes(dfg)

Return Vector{Symbol} of all unique factor types in factor graph.

lsfTypesDict(dfg)

Return ::Dict{DataType, Vector{Symbol}} of all unique factors types with labels in a factor graph.

findClosestTimestamp(setA, setB)

Find and return the closest timestamp from two sets of Tuples. Also return the minimum delta-time (::Nanosecond) and how many elements match from the two sets are separated by the minimum delta-time.

findPath(dfg, from::Symbol, to::Symbol; variableLabels, factorLabels, kwargs...)

Return the single shortest path between from and to. Errors if no path exists (use findPaths for graphs that may be disconnected).

Accepts the same restriction keywords as findPaths.

findPaths(dfg, from::Symbol, to::Symbol, k::Int; variableLabels, factorLabels, kwargs...)

Return the k shortest paths between from and to in the factor graph. Each result is a (path = Vector{Symbol}, dist) named tuple.

Optional keyword arguments restrict which variables and/or factors may appear on the path. When neither is given the full graph is used. When only one is provided the other defaults to all labels of that kind in dfg.

Typical usage with filters:

vars = listVariables(dfg; whereSolvable = >=(1))
facs = listFactors(dfg; whereSolvable = >=(1))
findPaths(dfg, :x1, :x5, 3; variableLabels = vars, factorLabels = facs)

See also: findPath, listVariables, listFactors

Speciallized function available to only GraphsDFG at this time.

Notes

• Has option for various types of filters (increases memory usage)

Example

using IncrementalInference

# canonical example graph as example
fg = generateGraph_Kaess()

@show path = findShortestPathDijkstra(fg, :x1, :x3)
@show isVariable.(fg, path)
@show isFactor.(fg, path)

DevNotes

• TODO expand to other AbstractDFG entities.
• TODO use of filter resource consumption can be improved.

Related

Graphs.dijkstra_shortest_paths

findVariablesNearTimestamp(
    dfg,
    query_timestamp;
    whereLabel,
    whereTags,
    whereSolvable,
    number
)

Find and return nearest variable labels per delta time. Function will filter on regexFilter, tags, and solvable.

Notes

• Returns Vector{Tuple{Vector{Symbol}, Nanosecond}}

DevNotes:

• TODO number should allow returning more than one for k-nearest matches.
• Future versions likely will require some optimization around the internal getVariable call.
  • Perhaps a dedicated/efficient getVariableTimestamp for all DFG flavors.

Related

ls, listVariables, findClosestTimestamp

Checks if the graph is fully connected, returns true if so. Implement isConnected(dfg::AbstractDFG)

listNeighborhood(dfg, label, distance; filters...)

Build a list of all unique neighbors inside 'distance'. Neighbors can be filtered by using keyword arguments, eg. [whereTags] and [whereSolvable]. Filters are applied to final neighborhood result.

Notes

• Returns a tuple (variableLabels, factorLabels), where each element is a Vector{Symbol}.

Related:

• getSubgraph
• mergeGraph!

Retrieve a list of labels of the immediate neighbors around a given variable or factor specified by its label. Implement listNeighbors(dfg::AbstractDFG, label::Symbol; whereSolvable, whereTags)

Remove tags from an agent's tag set (setdiff).

Remove tags from a factor's tag set (setdiff).

Remove tags from the graph-root tag set (setdiff).

Remove tags from a variable's tag set (setdiff).

Check whether an agent has all of the given tags.

Check whether a factor has all of the given tags.

Check whether the graph root has all of the given tags.

Check whether a variable has all of the given tags.

List all tags on an agent.

List all tags on a factor.

List all tags on the graph root.

List all tags on a variable.

Merge tags into an agent's tag set (union).

Merge tags into a factor's tag set (union).

Merge tags into the graph-root tag set (union).

Merge tags into a variable's tag set (union).

addBloblet!(node, bloblet)

addBloblets!(node, bloblets)

deleteBloblet!(node, label)

getBloblet(node, label)

getBloblets(node)

hasBloblet(node, label)

Check if a Bloblet with the given label exists on the node.

listBloblets(node)

List all Bloblet keys for a variable label in dfg

mergeBloblet!(node, bloblet)

Add a VariableDFG to a DFG. Implement addVariable!(dfg::AbstractDFG, variable::AbstractGraphVariable)

Add a Vector{VariableDFG} to a DFG. Implement addVariables!(dfg::AbstractDFG, variables::Vector{<:AbstractGraphVariable})

Delete a VariableDFG from the DFG. Implement deleteVariable!(dfg::AbstractDFG, label::Symbol)

Interface function to return the variableType dimension of an StateType, extend this function for all Types<:StateType.

getManifold(_)

Interface function to return the <:ManifoldsBase.AbstractManifold object of variableType<:StateType.

Interface function to return the user provided identity point for this StateType manifold, extend this function for all Types<:StateType.

Notes

• Used in transition period for Serialization. This function will likely be changed or deprecated entirely.

Interface function to return the manifold point type of an StateType, extend this function for all Types<:StateType.

getSolvedCount(v)

Get the number of times a variable has been inferred – i.e. solvedCount.

Related

isSolved, setSolvedCount!

getStateKind(_)

Get the kind of the variable's state, eg. Pose2, Point3, etc. as an instance of StateType.

Get a VariableDFG from a DFG using its label. Implement getVariable(dfg::AbstractDFG, label::Symbol)

Get a VariableSkeleton from a DFG.

Get a VariableSummary from a DFG.

Get the variables in the DFG as a Vector, supporting various filters.

Keyword arguments

• whereSolvable: Optional function to filter on the solvable property, eg >=(1).
• whereLabel: Optional function to filter on label e.g., contains(r"x1").
• whereTags: Optional function to filter on tags, eg. ⊇([:POSE]).
• whereType: Optional function to filter on the variable type.

Returns

• Vector{<:AbstractGraphVariable} matching the filters.

See also: listVariables, ls

Get the variables from a DFG as a Vector{VariableSkeleton}.

Get the variables from a DFG as a Vector{VariableSummary}.

True if the variable exists in the graph. Implement hasVariable(dfg::AbstractDFG, label::Symbol)

isInitialized(var)
isInitialized(var, key)

Returns state of variable data .initialized flag.

Notes:

• used by both factor graph variable and Bayes tree clique logic.

isMarginalized(vert)
isMarginalized(vert, solveKey)

Return ::Bool on whether this variable has been marginalized.

Notes:

• State default solveKey=:default

isSolved(v)

Boolean on whether the variable has been solved.

Related

getSolved, setSolved!

Get a list of labels of the DFGVariables in the graph. Supports optional arguments to filter the variables returned.

Notes

• Returns ::Vector{Symbol}

Example

listVariables(dfg)

See also: ls

Merge a variable into the DFG. If a variable with the same label exists, it will be overwritten; otherwise, the variable will be added to the graph. Implement mergeVariable!(dfg::AbstractDFG, variable::AbstractGraphVariable)

Merge a vector of variables into the DFG. If a variable with the same label exists, it will be overwritten; otherwise, the variable will be added to the graph. Implement mergeVariables!(dfg::AbstractDFG, variables::Vector{<:AbstractGraphVariable})

setMarginalized!(vnd, val)

Mark a variable as marginalized true or false.

setSolvedCount!(v, val)

Update/set the solveCount value.

Related

getSolved, isSolved

_getPriorType(T)

If you know a variable is ::Type{<:Pose2} but want to find its default prior ::Type{<:PriorPose2}.

Assumptions

• The prior type will be defined in the same module as the variable type.
• Not exported per default, but can be used with knowledge of the caveats.

Example

using RoME
@assert RoME.PriorPose2 == DFG._getPriorType(Pose2)

Add a FactorDFG to a DFG. Implement addFactor!(dfg::AbstractDFG, factor::AbstractGraphFactor)

Add a Vector{FactorDFG} to a DFG.

Delete a FactorDFG from the DFG using its label. Implement deleteFactor!(dfg::AbstractDFG, label::Symbol)

Delete Factors from the DFG using their labels or filters.

getCache(f)

Return the solver cache for a factor, which is used to store intermediate results during the solving process. This is useful for caching results that can be reused across multiple solves, such as Jacobians or other computed values.

Get a FactorDFG from a DFG using its label. Implement getFactor(dfg::AbstractDFG, label::Symbol)

List the DFGFactors in the DFG. Optionally specify a label regular expression to retrieves a subset of the factors.

Get the skeleton factors from a DFG as a Vector{FactorSkeleton}.

getObservation(f)

Return the observation of a factor, which is the user-defined data structure that contains the information about the factor, such as the measurement, prior, or relative pose.

getRecipehyper(f)

Return the hyperparameters associated with a factor in the factor graph. These hyperparameters may include settings such as multi-hypothesis handling, null hypothesis thresholds, and inflation factors that influence the behavior of the factor during optimization.

getRecipestate(f)

Return factor recipe state from factor graph.

getVariableOrder(fct)

Get the variable ordering for this factor. Should be equivalent to listNeighbors unless something was deleted in the graph.

True if the factor exists in the graph. Implement hasFactor(dfg::AbstractDFG, label::Symbol)

isPrior(_)
isPrior(_)

Return ::Bool on whether given factor fc::Symbol is a prior in factor graph dfg.

Get a list of the labels of the DFGFactors in the DFG. Optionally specify a label regular expression to retrieves a subset of the factors.

Merge a factor into the DFG. If a factor with the same label exists, it will be overwritten; otherwise, the factor will be added to the graph. Implement mergeFactor!(dfg::AbstractDFG, factor::AbstractGraphFactor)

setCache!(f, solvercache)

Set the solver cache for a factor, which is used to store intermediate results during the solving process. This is useful for caching results that can be reused across multiple solves, such as Jacobians or other computed values.

@defObservationType StructName factortype<:AbstractObservation manifolds<:AbstractManifold

A macro to create a new factor function with name StructName and manifold. Note that the manifold is an object and must be a subtype of ManifoldsBase.AbstractManifold. See documentation in Manifolds.jl on making your own.

Example:

DFG.@defObservationType Pose2Pose2 RelativeObservation SpecialEuclideanGroup(2)

Add variable solver data, errors if it already exists.

Add variable States by calling addState!. NOTE: If an error occurs while adding one of the states, previously added states will not be rolled back.

Delete the variable State by label, returns the number of deleted elements.

Delete variable States by label, returns the number of deleted elements.

Get the variable State for a given state label.

Get all the variable States for a given variable label.

True if the variable has a state with the given label.

List all the variable state labels.

Merge the variable state to the variable if it exists, otherwise add it.

Related mergeStates!

Merge variable states to the variable if they exist, otherwise add them.

Add an agent to the DFG.

Add multiple agents to the DFG.

Delete an agent by label.

Delete multiple agents by label.

Get an agent by label.

Get all agents in the DFG.

Check whether an agent with the given label exists.

List all agent labels in the DFG.

Merge an agent into the DFG (add or update).

Merge multiple agents into the DFG.

exists(dfg, label)

True if a variable or factor with label exists in the graph.

getSubgraph(, dfg, variableFactorLabels; ...)
getSubgraph(
    ,
    dfg,
    variableFactorLabels,
    distance;
    whereSolvable,
    whereTags,
    graphLabel,
    solvable,
    kwargs...
)

Build a deep subgraph copy from the DFG given a list of variables and factors and an optional distance. Note: Orphaned factors (where the subgraph does not contain all the related variables) are not returned. Related:

• listNeighborhood
• mergeGraph!

Dev Notes

• Bulk vs node for node: a list of labels are compiled and the sugraph is copied in bulk.

Return whether sym::Symbol represents a factor vertex in the graph DFG. Checks whether it both exists in the graph and is a factor. (If you rather want a quicker for type, just do node isa FactorDFG) Implement isFactor(dfg::AbstractDFG, label::Symbol)

Return whether sym::Symbol represents a variable vertex in the graph DFG. Checks whether it both exists in the graph and is a variable. (If you rather want a quick for type, just do node isa VariableDFG) Implement isVariable(dfg::AbstractDFG, label::Symbol)

mergeGraph!(destDFG, srcDFG)

Merge the source DFG into the destination DFG cascading down the hierarchy of DFG nodes. Merge rules:

• Variables, Factors, Agents, and Graphroot, with the same label are merged if they are equal.
  • On conflicts, a MergeConflictError is thrown.
  • Child nodes (eg. tags, Bloblets, Blobentries, States, etc.) are using merge!.
• The Blobprovider links are merged provided they point to the same Blobprovider.
  • On conflicts, a MergeConflictError is thrown.

printFactor(dfg, sym; kwargs...)

Display and return to console the user factor identified by tag name.

printVariable(dfg, sym; kwargs...)

Display the content of State to console for a given factor graph and variable tag::Symbol.

Dev Notes

• TODO split as two show macros between AMP and DFG

compareAll(Al, Bl; show, skip)

Recursively compare the all fields of T that are not in skip for objects Al and Bl.

TODO > add to func_ref.md

compareAllVariables(fgA, fgB; skip, show, skipsamples)

Compare all variables in both ::FactorGraphs A and B.

Notes

• A and B should all the same variables and factors.

Related:

compareFactorGraphs, compareSimilarVariables, compareVariable, ls

compareFactor(A, B; show, skip, skipsamples, skipcompute)

Compare that all fields are the same in a ::FactorGraph factor.

DevNotes

• TODO getSolverData(A).fnc.varValsAll / varidx are only defined downstream, so should should this function not be in IIF?

compareFactorGraphs(
    fgA,
    fgB;
    skipsamples,
    skipcompute,
    skip,
    show
)

Compare and return if two factor graph objects are the same, by comparing similar variables and factors.

Notes:

• Default items to skip with skipsamples, skipcompute.
• User defined fields to skip can be specified with skip::Vector{Symbol}.
• To enable debug messages for viewing which fields are not the same:
  • https://stackoverflow.com/questions/53548681/how-to-enable-debugging-messages-in-juno-julia-editor

Related:

compareSimilarVariables, compareSimilarFactors, compareAllVariables, ls.

compareFields(Al, Bl; show, skip)

Compare the all fields of T that are not in skip for objects Al and Bl and returns ::Bool.

TODO > add to func_ref.md

compareSimilarFactors(
    fgA,
    fgB;
    skipsamples,
    skipcompute,
    skip,
    show
)

Compare similar factors between ::FactorGraphs A and B.

Related:

compareFactorGraphs, compareSimilarVariables, compareAllVariables, ls.

compareSimilarVariables(fgA, fgB; skip, show, skipsamples)

Compare similar labels between ::FactorGraphs A and B.

Notes

• At least one variable label should exist in both A and B.

Related:

compareFactorGraphs, compareAllVariables, compareSimilarFactors, compareVariable, ls.

compareVariable(A, B; skip, show, skipsamples)

Compare that all fields are the same in a ::FactorGraph variable.

DFGJSONStyle <: JSON.JSONStyle

Custom JSON serialization style used throughout DFG for JSON.json / JSON.parse.

This style adds handling for types that don't round-trip through plain JSON: Complex, SArray, ArrayPartition, RefValue{Int}, TimeDateZone, etc.

Polymorphic abstract types — pack/unpack

Abstract types like AbstractBlobprovider and AbstractObservation are serialized through the Packed envelope. Each concrete subtype can define a lightweight "packed" companion struct and overload pack / unpack. The default pack(x) = x works for structs whose fields are all plain data.

When a type contains non-serializable fields (clients, connections, caches), define a packed companion:

struct PackedMytype
    label::Symbol
end
DFG.pack(s::Mytype) = PackedMytype(s.label)
DFG.unpack(p::PackedMytype) = Mytype(reconnect_client(), p.label)

JSON emitted with style = DFGJSONStyle() embeds a "type" header so the deserializer can locate the packed type and call unpack.

Usage

json_str = JSON.json(value; style = DFGJSONStyle())
value    = JSON.parse(json_str, T; style = DFGJSONStyle())

See also: pack, unpack, Packed, @packed

Packed{T}(type::TypeMetadata, packed::T)
Packed(x)

Serialization envelope that pairs a value with its TypeMetadata.

Calling Packed(x) runs the full pipeline:

1. packed_x = pack(x) — convert x to its serialization-safe form.
2. TypeMetadata(typeof(packed_x)) — snapshot the packed type's identity.
3. Bundle both into Packed{typeof(packed_x)}(metadata, packed_x).

During serialization (lower): the packed struct's fields are flattened into an OrderedDict alongside a :type key holding the metadata.

During deserialization (lift): resolvePackedType reads the :type key to identify the packed type, StructUtils.make builds it, and unpack reconstructs the original live type.

See also: pack, unpack, TypeMetadata, resolvePackedType

TypeMetadata(pkg, name, version)
TypeMetadata(::Type{T})

Metadata header embedded in every Packed envelope.

Stores the top-level package name, the struct name, and the package version so the deserializer can resolve the correct concrete type at load time via resolvePackedType.

Generally you never construct this manually — Packed(x) does it for you.

pack(x) -> packed_x

Convert x into a serialization-friendly "packed" form.

The default method is the identity (pack(x) = x), meaning types whose fields are all plain data (numbers, strings, arrays, nested structs of the same) need no special treatment — they serialize as-is through StructUtils.jl.

Overload pack when a type contains non-serializable fields such as network connections, open file handles, live caches, or opaque foreign objects. The pattern is:

1. Define a "packed" companion struct holding only the serializable subset.
2. Overload pack to project the live type onto the packed struct.
3. Overload unpack to reconstruct the live type from the packed struct.

The Packed wrapper calls pack automatically during serialization, embedding a TypeMetadata header so the deserializer can locate the packed type.

Example

# Live type — `client` is an HTTP connection, not serializable
struct MyRemoteStore <: DFG.AbstractBlobprovider
    client::HTTPClient
    label::Symbol
end

# Packed form — only the serializable fields
struct PackedMyRemoteStore
    label::Symbol
end

DFG.pack(s::MyRemoteStore) = PackedMyRemoteStore(s.label)
DFG.unpack(p::PackedMyRemoteStore) = MyRemoteStore(find_active_client(), p.label)

The JSON output will contain "type": {"pkg": "MyPkg", "name": "PackedMyRemoteStore", ...} so the deserializer resolves the packed type, calls StructUtils.make(PackedMyRemoteStore, json), and then calls unpack(::PackedMyRemoteStore) to produce the live MyRemoteStore.

See also: unpack, Packed, TypeMetadata, DFGJSONStyle

pack_lower(x)

Convenience: pack then lower in one call. Equivalent to StructUtils.lower(Packed(x)).

resolvePackedType(json_obj) -> Type{Packed{T}}

Read the type metadata from a JSON object and resolve the corresponding Packed{T} type. Called automatically by the @choosetype machinery during deserialization of abstract types (e.g. AbstractBlobprovider, Packed).

The resolver requires the owning module to be loaded in Main.

unpack(packed_x) -> x

Reconstruct a live object from its packed (serialization-safe) form.

The default method is the identity (unpack(x) = x). Overload it for each packed companion type created alongside a pack overload.

During deserialization the pipeline is:

JSON bytes  →  StructUtils.make(PackedT, data)  →  unpack(::PackedT)  →  live T

unpack is the place to reconnect non-serializable resources (clients, file handles, caches) that were stripped by pack.

Example

DFG.unpack(p::PackedMyRemoteStore) = MyRemoteStore(find_active_client(), p.label)

See also: pack, Packed

DFG.@packed

Macro annotation for DFG serialization metadata on struct fields. Expands to (lower = DFG.Packed, choosetype = DFG.resolvePackedType) for use with StructTypes.jl field annotations.

Used to mark belief fields in factor types for serialization through the DFG packing system. The lower function converts the field to a Packed wrapper during serialization, and choosetype resolves the correct type during deserialization.

Usage

Use with the & operator in @kwarg or @tags struct definitions:

@kwarg struct Pose2Point2Range{T} <: AbstractRelativeObservation
    Z::T & DFG.@packed
    partial::Tuple{Int, Int} = (1, 2)
end

This is equivalent to writing:

@kwarg struct Pose2Point2Range{T} <: AbstractRelativeObservation
    Z::T & (lower = DFG.Packed, choosetype = DFG.resolvePackedType)
    partial::Tuple{Int, Int} = (1, 2)
end

See also: Packed, pack, unpack, resolvePackedType