Blobs and Blobentries
Types
DistributedFactorGraphs.Blobentry — Typestruct BlobentryA Blobentry is a small about of structured data that holds reference information to find an actual blob. Many Blobentrys can exist on different graph nodes spanning Agents and Factor Graphs which can all reference the same Blob.
Notes:
blobIds should be unique within a blobstore and are immutable.
DistributedFactorGraphs.AbstractBlobstore — TypeAbstractBlobstore{T}Abstract supertype for all blobstore implementations.
Usage
Subtypes of AbstractBlobstore{T} must implement the required interface for blob storage and retrieval, such as:
add!(store, blobId, blob): Add a new blob to the store.get(store, blobId): Retrieve a blob by its ID.list(store): List all blob IDs in the store.
The parameter T represents the type of blobs stored (e.g., Vector{UInt8} or a custom Blob type).
See concrete implementations for details.
Design Notes
blobIdis not considered unique across blobstores with different labels only within a single blobstore.- We cannot guarantee that
blobIdis unique across different blobstores with the same label and this is up to the end user. - Within a single blobstore
addBlob!will fail if there is a UUID collision. - TODO: We should consider using uuid7 for
blobIds (requires jl v1.12). Blobstroresare identified by alabel::Symbol, which allows for multiple blobstores to coexist in the same system.
TODO: If we want to make the blobId=>Blob pair immutable:
- We can use the tombstone pattern to mark a blob as deleted. See FolderStore in PR#TODO.
Design goal: all Blobstores with the same label can contain the same blobId=>Blob pair and the blobs should be identical since they are immutable.
Functions
DistributedFactorGraphs.addBlobentry! — MethodaddBlobentry!(var, entry)
Add a Blobentry to a variable Should be extended if DFG variable is not returned by reference.
Also see: getBlobentry, addBlob!, mergeBlobentry!
DistributedFactorGraphs.buildSourceString — MethodbuildSourceString(dfg, label)
Function to generate source string - agentLabel|graphLabel|varLabel
DistributedFactorGraphs.deleteBlobentry! — MethoddeleteBlobentry!(var, key)
Delete a Blobentry from the factor graph variable.
Notes:
- This doesn't remove the associated
Blobfrom any Blobstores.
DistributedFactorGraphs.getBlobentries — MethodgetBlobentries(v)
Get blob entries, returns a Vector{Blobentry}.
DistributedFactorGraphs.getBlobentry — MethodDistributedFactorGraphs.getfirstBlobentry — MethodgetfirstBlobentry(
v;
labelFilter,
blobIdFilter,
sortby,
sortlt
)
Finds and returns the first blob entry that matches the filter. The result is sorted by sortby[=getLabel] and sortlt[=natural_lt] before returning the first entry. Also see: getBlobentry
DistributedFactorGraphs.hasBlobentry — MethodhasBlobentry(v, blobLabel)
Does a blob entry exist with blobLabel.
DistributedFactorGraphs.incrDataLabelSuffix — MethodincrDataLabelSuffix(dfg, vla, bllb; datalabel)
If the blob label datalabel already exists, then this function will return the name datalabel_1. If the blob label datalabel_1 already exists, then this function will return the name datalabel_2.
DistributedFactorGraphs.listBlobentries — MethodlistBlobentries(var)
List the blob entries associated with a particular variable.
DistributedFactorGraphs.listBlobentrySequence — FunctionlistBlobentrySequence(dfg, lb, pattern)
listBlobentrySequence(dfg, lb, pattern, _sort)
List a collection of blob entries per variable that match a particular pattern::Regex.
Notes
- Optional sort function argument, default is unsorted.
- Likely use of
sortDFGfor basic Symbol sorting.
- Likely use of
Example
listBlobentrySequence(fg, :x0, r"IMG_CENTER", sortDFG)
15-element Vector{Symbol}:
:IMG_CENTER_21676
:IMG_CENTER_21677
:IMG_CENTER_21678
:IMG_CENTER_21679
...DistributedFactorGraphs.mergeBlobentry! — MethodmergeBlobentry!(var, bde)
Update a Blobentry in the factor graph. If the Blobentry does not exist, it will be added. Notes:
DistributedFactorGraphs.addBlob! — FunctionAdds a blob to the blob store or dfg with the blobId.
Related addBlobentry! Implement addBlob!(store::AbstractBlobstore, blobId::UUID, data)
addBlob!(dfg, entry, data)addBlob!(store, entry, data)addBlob!(store, data)addBlob!(store, blobId, data)addBlob!(store, blobId, data)addBlob!(store, blobId, linkfile)addBlob!(store, blobId, blob)addBlob!(store, data, _)
addBlob!(store, blobId, data, _)defined at deprecated.jl:103.
DistributedFactorGraphs.deleteBlob! — FunctionDelete a blob from the blob store or dfg with the given entry.
Related deleteBlobentry! Implement deleteBlob!(store::AbstractBlobstore, blobId::UUID)
deleteBlob!(dfg, entry)deleteBlob!(store, entry)deleteBlob!(store, blobId)deleteBlob!(store, blobId)deleteBlob!(store)deleteBlob!(store, _)deleteBlob!(store, _)deleteBlob!(store, blobId)DistributedFactorGraphs.getBlob — FunctionGet the data blob for the specified blobstore or dfg.
Related getBlobentry Implement getBlob(store::AbstractBlobstore, blobId::UUID)
getBlob(dfg, entry)getBlob(store, entry)getBlob(store, blobId)getBlob(store, blobId)getBlob(store, blobId)getBlob(store, blobId)DistributedFactorGraphs.listBlobs — FunctionList all blobIds in the blob store. Implement listBlobs(store::AbstractBlobstore)
DistributedFactorGraphs.packBlob — FunctionpackBlobConvert a file (JSON, JPG, PNG, BSON, LAS) to Vector{UInt8} for use as a Blob. Returns the blob and MIME type.
DistributedFactorGraphs.unpackBlob — FunctionunpackBlobConvert a Blob back to the origanal typ using the MIME type or DataFormat type.
DistributedFactorGraphs.deleteBlob_Agent! — FunctionConvenience wrapper to delete a Blob from a Blobstore and its Blobentry from an agent.
deleteBlob_Agent!(dfg, entry_label)DistributedFactorGraphs.deleteBlob_Graph! — FunctionConvenience wrapper to delete a Blob from a Blobstore and its Blobentry from a graph.
deleteBlob_Graph!(dfg, entry_label)DistributedFactorGraphs.deleteBlob_Variable! — FunctionConvenience wrapper to delete a Blob form a Blobstore and its Blobentry from a variable.
deleteBlob_Variable!(dfg, variable_label, entry_label)DistributedFactorGraphs.loadBlob_Agent — FunctionConvenience wrapper to load a Blob for a given agent and Blobentry label.
loadBlob_Agent(dfg, entry_label)DistributedFactorGraphs.loadBlob_Graph — FunctionConvenience wrapper to load a Blob for a given graph and Blobentry label.
loadBlob_Graph(dfg, entry_label)DistributedFactorGraphs.loadBlob_Variable — FunctionConvenience wrapper to load a Blob for a given variable and Blobentry label.
loadBlob_Variable(dfg, variable_label, entry_label)DistributedFactorGraphs.saveBlob_Agent! — FunctionConvenience wrapper to save a Blob to a Blobstore and a Blobentry to an agent.
saveBlob_Agent!(dfg, blob, entry)saveBlob_Agent!(
dfg,
blob,
entry_label,
blobstore;
blobentry_kwargs...
)
saveBlob_Agent!(dfg, blob, entry_label; ...)DistributedFactorGraphs.saveBlob_Graph! — FunctionConvenience wrapper to save a Blob to a Blobstore and a Blobentry to a graph.
saveBlob_Graph!(dfg, blob, entry)saveBlob_Graph!(
dfg,
blob,
entry_label,
blobstore;
blobentry_kwargs...
)
saveBlob_Graph!(dfg, blob, entry_label; ...)DistributedFactorGraphs.saveBlob_Variable! — FunctionConvenience wrapper to save a Blob to a Blobstore and a Blobentry to a variable.
saveBlob_Variable!(dfg, variable_label, blob, entry)saveBlob_Variable!(
dfg,
variable_label,
blob,
entry_label,
blobstore;
blobentry_kwargs...
)
saveBlob_Variable!(
dfg,
variable_label,
blob,
entry_label;
...
)