Blobs and Blobentries

Types

DistributedFactorGraphs.BlobentryType
struct Blobentry

A Blobentry is a small about of structured data that holds reference information to find an actual blob. Many Blobentrys can exist on different graph nodes spanning Agents and Factor Graphs which can all reference the same Blob.

Notes:

  • blobIds should be unique within a blobstore and are immutable.
source
DistributedFactorGraphs.AbstractBlobstoreType
AbstractBlobstore{T}

Abstract supertype for all blobstore implementations.

Usage

Subtypes of AbstractBlobstore{T} must implement the required interface for blob storage and retrieval, such as:

  • add!(store, blobId, blob): Add a new blob to the store.
  • get(store, blobId): Retrieve a blob by its ID.
  • list(store): List all blob IDs in the store.

The parameter T represents the type of blobs stored (e.g., Vector{UInt8} or a custom Blob type).

See concrete implementations for details.

Design Notes

  • blobId is not considered unique across blobstores with different labels only within a single blobstore.
  • We cannot guarantee that blobId is unique across different blobstores with the same label and this is up to the end user.
  • Within a single blobstore addBlob! will fail if there is a UUID collision.
  • TODO: We should consider using uuid7 for blobIds (requires jl v1.12).
  • Blobstroresare identified by a label::Symbol, which allows for multiple blobstores to coexist in the same system.

TODO: If we want to make the blobId=>Blob pair immutable:

  • We can use the tombstone pattern to mark a blob as deleted. See FolderStore in PR#TODO.

Design goal: all Blobstores with the same label can contain the same blobId=>Blob pair and the blobs should be identical since they are immutable.

source

Functions

DistributedFactorGraphs.deleteBlobentry!Method
deleteBlobentry!(var, key)

Delete a blob entry from the factor graph. Note this doesn't remove it from any data stores.

Notes:

  • users responsibility to delete data in db before deleting entry
source
DistributedFactorGraphs.getBlobentriesVariablesMethod
getBlobentriesVariables(
    dfg,
    bLblPattern;
    varList,
    dropEmpties
)

Get all blob entries matching a Regex pattern over variables

Notes

  • Use dropEmpties=true to not include empty lists in result.
  • Use keyword varList for which variables to search through.
source
DistributedFactorGraphs.incrDataLabelSuffixMethod
incrDataLabelSuffix(dfg, vla, bllb; datalabel)

If the blob label datalabel already exists, then this function will return the name datalabel_1. If the blob label datalabel_1 already exists, then this function will return the name datalabel_2.

source
DistributedFactorGraphs.listBlobentrySequenceFunction
listBlobentrySequence(dfg, lb, pattern)
listBlobentrySequence(dfg, lb, pattern, _sort)

List a collection of blob entries per variable that match a particular pattern::Regex.

Notes

  • Optional sort function argument, default is unsorted.
    • Likely use of sortDFG for basic Symbol sorting.

Example

listBlobentrySequence(fg, :x0, r"IMG_CENTER", sortDFG)
15-element Vector{Symbol}:
 :IMG_CENTER_21676
 :IMG_CENTER_21677
 :IMG_CENTER_21678
 :IMG_CENTER_21679
...
source
DistributedFactorGraphs.addBlob!Function

Adds a blob to the blob store or dfg with the blobId.

Related addBlobentry! Implement addBlob!(store::AbstractBlobstore, blobId::UUID, data)

addBlob!(dfg, entry, data)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:62.

addBlob!(store, entry, data)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:66.

addBlob!(store, data)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:71.

addBlob!(store, blobId, data)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:154.

addBlob!(store, blobId, data)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:218.

addBlob!(store, entry, linkfile)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:278.

addBlob!(store, blobId, linkfile)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:282.

addBlob!(store, blobId, blob)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:366.

addBlob!(store, data, _)
addBlob!(store, blobId, data, _)

defined at deprecated.jl:103.

source
DistributedFactorGraphs.deleteBlob!Function

Delete a blob from the blob store or dfg with the given entry.

Related deleteBlobentry! Implement deleteBlob!(store::AbstractBlobstore, blobId::UUID)

deleteBlob!(dfg, entry)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:82.

deleteBlob!(store, entry)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:86.

deleteBlob!(store, blobId)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:178.

deleteBlob!(store, blobId)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:233.

deleteBlob!(store)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:293.

deleteBlob!(store, _)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:297.

deleteBlob!(store, _)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:298.

deleteBlob!(store, blobId)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:381.

source
DistributedFactorGraphs.getBlobFunction

Get the data blob for the specified blobstore or dfg.

Related getBlobentry Implement getBlob(store::AbstractBlobstore, blobId::UUID)

getBlob(dfg, entry)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:51.

getBlob(store, entry)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:57.

getBlob(store, blobId)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:143.

getBlob(store, blobId)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:211.

getBlob(store, blobId)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:270.

getBlob(store, blobId)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/BlobStores.jl:359.

source
DistributedFactorGraphs.addData!Function

Add both a Blobentry and Blob to a distributed factor graph or Blobstore. Related addBlobentry!

addData!(dfg, label, entry, blob; hashfunction, checkhash)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/HelpersDataWrapEntryBlob.jl:124.

addData!(
    dfg,
    blobstore,
    label,
    entry,
    blob;
    hashfunction,
    checkhash
)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/HelpersDataWrapEntryBlob.jl:138.

addData!(
    dfg,
    blobstorekey,
    vLbl,
    bLbl,
    blob,
    timestamp;
    kwargs...
)
addData!(dfg, blobstorekey, vLbl, bLbl, blob; ...)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/HelpersDataWrapEntryBlob.jl:153.

addData!(
    dfg,
    blobstore,
    vLbl,
    bLbl,
    blob,
    timestamp;
    description,
    metadata,
    mimeType,
    id,
    blobId,
    originId,
    hashfunction
)
addData!(dfg, blobstore, vLbl, bLbl, blob; ...)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/HelpersDataWrapEntryBlob.jl:173.

addData!(
    dfg,
    blobstore,
    vLbl,
    blobLabel,
    blob,
    timestamp;
    description,
    metadata,
    mimeType,
    origin
)
addData!(dfg, blobstore, vLbl, blobLabel, blob; ...)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/HelpersDataWrapEntryBlob.jl:206.

source
DistributedFactorGraphs.deleteData!Function
source
DistributedFactorGraphs.getDataFunction

Get the blob entry and blob for the specified blobstore or dfg retured as a tuple. Related getBlobentry

getData(dfg, vlabel, key; hashfunction, checkhash, getlast)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/HelpersDataWrapEntryBlob.jl:81.

getData(
    dfg,
    blobstore,
    label,
    key;
    hashfunction,
    checkhash,
    getlast
)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/HelpersDataWrapEntryBlob.jl:108.

source
DistributedFactorGraphs.updateData!Function

Update a blob entry or blob to the blob store or dfg. Related mergeBlobentry!

updateData!(
    dfg,
    label,
    entry,
    blob;
    hashfunction,
    checkhash
)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/HelpersDataWrapEntryBlob.jl:240.

updateData!(
    dfg,
    blobstore,
    label,
    entry,
    blob;
    hashfunction
)

defined at /home/runner/work/DistributedFactorGraphs.jl/DistributedFactorGraphs.jl/src/DataBlobs/services/HelpersDataWrapEntryBlob.jl:255.

source