Object Stores

Modules
	Object Stores management
	Object Store management

Functions
NATS_EXTERN natsStatus	objStoreMeta_Init (objStoreMeta *meta)
	Initializes an object store meta structure.
NATS_EXTERN natsStatus	objStore_Put (objStorePut **new_put, objStore *obs, objStoreMeta *meta)
	Initiates a session to put bytes into an object.
NATS_EXTERN natsStatus	objStorePut_Add (objStorePut *put, const void *data, int dataLen)
	Add data to the object.
NATS_EXTERN natsStatus	objStorePut_Complete (objStoreInfo **new_info, objStorePut *put, int64_t timeout)
	Complete a put operation.
NATS_EXTERN void	objStorePut_Destroy (objStorePut *put)
	Destroys the object store's put object.
NATS_EXTERN natsStatus	objStore_PutString (objStoreInfo **new_info, objStore *obs, const char *name, const char *data)
	Put a string into this object.
NATS_EXTERN natsStatus	objStore_PutBytes (objStoreInfo **new_info, objStore *obs, const char *name, const void *data, int dataLen)
	Put bytes into this object.
NATS_EXTERN natsStatus	objStore_PutFile (objStoreInfo **new_info, objStore *obs, const char *fileName)
	Put the content of a file into this object.
NATS_EXTERN natsStatus	objStore_Get (objStoreGet **new_get, objStore *obs, const char *name, objStoreOptions *opts)
	Pull the named object from the object store.
NATS_EXTERN natsStatus	objStoreGet_Info (const objStoreInfo **new_info, objStoreGet *get)
	Returns a handle to the information object own by the objStoreGet object.
NATS_EXTERN natsStatus	objStoreGet_Read (bool *done, void **new_data, int *dataLen, objStoreGet *get, int64_t timeout)
	Returns some bytes of the pulled object.
NATS_EXTERN natsStatus	objStoreGet_ReadAll (void **new_data, int *dataLen, objStoreGet *get, int64_t timeout)
	Returns the remaining bytes of the pulled object.
NATS_EXTERN void	objStoreGet_Destroy (objStoreGet *get)
	Destroys the object store's get object.
NATS_EXTERN natsStatus	objStore_GetString (char **new_str, objStore *obs, const char *name, objStoreOptions *opts)
	Pull the named object from the object store and return it as a string.
NATS_EXTERN natsStatus	objStore_GetBytes (void **new_data, int *dataLen, objStore *obs, const char *name, objStoreOptions *opts)
	Pull the named object from the object store and return it as a byte array.
NATS_EXTERN natsStatus	objStore_GetFile (objStore *obs, const char *name, const char *fileName, objStoreOptions *opts)
	Pull the named object from the object store and place it into a file.

Detailed Description

Object Stores offer a straightforward method for storing large objects within JetStream. These stores are backed by a specially configured streams, designed to efficiently and compactly store these objects.

The Object Store, also known as a bucket, enables the execution of various operations:

• create an object store
• update an object store
• get an object store
• delete an object store
• list all objects in a bucket
• watch for changes on objects in a bucket
• create links to other objects or other buckets

Warning

Object Stores needs TLS to be able to generate and check SHA256 object digests. If the library is compiled without TLS support, some of the features will still work, but "put" and "get" operations will fail.

Function Documentation

objStoreMeta_Init()

NATS_EXTERN natsStatus objStoreMeta_Init

(

objStoreMeta *

meta

)

Typically, the user will declare an objStoreMeta variable on the stack and call this function to initializes the structure, set some fields and pass it to an API.

objStoreMeta meta;
 
objStoreMeta_Init(&meta);
meta.Name = "test";
s = objStore_Put(&put, obs, &meta);
...

See also

objStore_Put

Parameters

meta	the pointer to the objStoreMeta structure to initialize.

objStore_Put()

NATS_EXTERN natsStatus objStore_Put	(	objStorePut **	new_put,
		objStore *	obs,
		objStoreMeta *	meta
	)

If the object already exists, it will be overwritten. The object name is required and taken from the objStoreMeta.Name field.

On success, an objStorePut object will be returned. This should then be passed to the functions to add and complete the put operation.

Note

The returned objStorePut object needs be destroyed using objStorePut_Destroy when no longer needed to free allocated memory.

objStoreMeta meta;
objStorePut  *put  = NULL;
objStoreInfo *info = NULL;
 
objStoreMeta_Init(&meta);
meta.Name = "test";
s = objStore_Put(&put, obs, &meta);
if (s == NATS_OK)
{
     natsStatus addSts = NATS_OK;
 
     while ((addSts == NATS_OK) && hasMoreDataToAdd)
     {
             // Get some data from somewhere...
             addSts = objStorePut_Add(put, data, dataLen);
             // If data needs to be freed, it is safe to do it after the "Add" call.
             ...
     }
     // We are done adding data, call "Complete". We should invoke this even
     // if there were issues adding data. The call will then purge partial chunks.
     // The last param is the timeout for operation to complete, here using 3 seconds.
     s = objStorePut_Complete(&info, put, 3000);
}
// At this point, the "put" operation is complete. On success, we would have an `info`
// variable that can be inspected. If not needed, `NULL` should have been passed to
// the `objStorePut_Complete` call.
if (s == NATS_OK)
{
     // Inspect the info if needed.
}
// Now we need to destroy both object. No need to check for `NULL` (as long as you
properly initialized them to `NULL`) since the calls internally check for that.
objStoreInfo_Destroy(info);
objStorePut_Destroy(put);
...

See also

objStoreMeta_Init

objStorePut_Add

objStorePut_Complete

objStorePut_Destroy

Parameters

new_put	the location where to store the pointer to the objStorePut object.
obs	the pointer to the objStore object.
meta	the pointer to the objStoreMeta object.

objStorePut_Add()

NATS_EXTERN natsStatus objStorePut_Add	(	objStorePut *	put,
		const void *	data,
		int	dataLen
	)

After successfully calling objStore_Put, the returned objStorePut object is used by this function to add bytes to the object. When there is no more data to add, call objStorePut_Complete to complete the operation.

If an error occurs, invoking this function again will immediately error out. The only outcome is to call objStorePut_Complete to undo the partial additions and free the object.

See objStore_Put for an example on how to use the set of APIs.

See also

objStore_Put

objStorePut_Complete

Parameters

put	the pointer to the objStorePut object.
data	the pointer to the byte array to add to the object.
dataLen	the number of bytes of the byte array to add.

objStorePut_Complete()

NATS_EXTERN natsStatus objStorePut_Complete	(	objStoreInfo **	new_info,
		objStorePut *	put,
		int64_t	timeout
	)

After adding data, this call will complete the operation and on success return the objStoreInfo (if desired).

If there were errors adding the data, this function will purge the partial data added.

Note

The returned objStoreInfo object needs be destroyed using objStoreInfo_Destroy when no longer needed to free allocated memory.

Parameters

new_info	the location where to store the pointer to the objStoreInfo object, or NULL if not needed.
put	the pointer to the objStorePut object.
timeout	the amount of time (in milliseconds) to wait for the operation to complete.

objStorePut_Destroy()

NATS_EXTERN void objStorePut_Destroy

(

objStorePut *

put

)

Releases memory allocated for this objStorePut object.

Note

If there were errors and the put operation did not complete, this function will attempt to purge the data that was partially added.

Parameters

put	the pointer to the objStorePut object.

objStore_PutString()

NATS_EXTERN natsStatus objStore_PutString	(	objStoreInfo **	new_info,
		objStore *	obs,
		const char *	name,
		const char *	data
	)

This is a convenience function to put a string into this object store under the given name.

An objStoreInfo will be returned, containing the object's metadata, digest and instance information.

Note

The returned objStoreInfo object needs be destroyed using objStoreInfo_Destroy when no longer needed to free allocated memory.

See also

objStoreInfo_Destroy

Parameters

new_info	the location where to store the pointer to the objStoreInfo object, or NULL if not needed.
obs	the pointer to the objStore object.
name	the name of the object to put the string into.
data	the string to put into the object.

objStore_PutBytes()

NATS_EXTERN natsStatus objStore_PutBytes	(	objStoreInfo **	new_info,
		objStore *	obs,
		const char *	name,
		const void *	data,
		int	dataLen
	)

This is a convenience function to put bytes into this object store under the given name.

An objStoreInfo will be returned, containing the object's metadata, digest and instance information.

Note

The returned objStoreInfo object needs be destroyed using objStoreInfo_Destroy when no longer needed to free allocated memory.

See also

objStoreInfo_Destroy

Parameters

new_info	the location where to store the pointer to the objStoreInfo object, or NULL if not needed.
obs	the pointer to the objStore object.
name	the name of the object to put the string into.
data	the bytes to put into the object.
dataLen	the number of bytes to put into the object.

objStore_PutFile()

NATS_EXTERN natsStatus objStore_PutFile	(	objStoreInfo **	new_info,
		objStore *	obs,
		const char *	fileName
	)

This is a convenience function to put a file content into this object store. The name of the object will be the path of the file.

Note

The returned objStoreInfo object needs be destroyed using objStoreInfo_Destroy when no longer needed to free allocated memory.

See also

objStoreInfo_Destroy

Parameters

new_info	the location where to store the pointer to the objStoreInfo object, or NULL if not needed.
obs	the pointer to the objStore object.
fileName	the name of the file whose content will be put into the object.

objStore_Get()

NATS_EXTERN natsStatus objStore_Get	(	objStoreGet **	new_get,
		objStore *	obs,
		const char *	name,
		objStoreOptions *	opts
	)

If the object does not exist, NATS_NOT_FOUND will be returned.

The returned objStoreGet object will contain the object's metadata and the user can call objStoreGet_Read to get some bytes as they become available, or objStoreGet_ReadAll to get the whole object in one call.

The option objStoreOptions.ShowDeleted can be provided to return an object even if it was marked as deleted.

Note

The returned objStoreGet object needs be destroyed using objStoreGet_Destroy when no longer needed to free allocated memory.

Here is an example to read data by chunks:

objStoreGet *get = NULL;
 
s = objStore_Get(&get, obs, "myobject", NULL);
if (s == NATS_OK)
{
     objStoreInfo    *info   = NULL;
     bool            done    = false;
 
     // We can inspect the info if we chose to do so.
     s = objStoreGet_Info(&info);
     if (s == NATS_OK)
     {
             // Inspect the info...
     }
     // DO NOT destroy the info, it belongs to the `objStoreGet` object.
 
     // Get data in chunks.
     while ((s == NATS_OK) && !done)
     {
             void *data = NULL;
             int  len   = 0;
 
             // Read some data. The last param is a timeout (in milliseconds) to wait
             // for some data to become available.
             s = objStoreGet_Read(&done, &data, &len, get, 4000);
             if (s == NATS_OK)
             {
                     // Do something with the data
                     ...
                     // It is the user responsibility to free the memory.
                     free(data);
             }
     }
     // At this point, all data for the object has been retrieved.
}
// Now we need to destroy the object. No need to check for `NULL` (as long as you
properly initialized it to `NULL`) since the call internally check for that.
objStoreGet_Destroy(get);
...

Here is an example to read data all at once. Unless the user needs to inspect the object store information's object prior to read the data, the convenience functtion objStore_GetBytes will be equivalent much simpler to use.

objStoreGet *get = NULL;
 
s = objStore_Get(&get, obs, "myobject", NULL);
if (s == NATS_OK)
{
     objStoreInfo *info = NULL;
 
     // We can inspect the info if we chose to do so.
     s = objStoreGet_Info(&info);
     if (s == NATS_OK)
     {
             // Inspect the info...
     }
     // DO NOT destroy the info, it belongs to the `objStoreGet` object.
 
     if (s == NATS_OK)
     {
             void    *data   = NULL;
             int     len     = 0;
 
             // The last param is a timeout (in milliseconds) to wait
             // for some data to become available. Here we are using 10 seconds.
             s = objStoreGet_ReadAll(&data, &len, get, 10000);
             if (s == NATS_OK)
             {
                     // Do something with the data
                     ...
                     // It is the user responsibility to free the memory.
                     free(data);
             }
     }
}
// Now we need to destroy the object. No need to check for `NULL` (as long as you
properly initialized it to `NULL`) since the call internally check for that.
objStoreGet_Destroy(get);
...

See also

objStoreGet_Read

objStoreGet_ReadAll

objStoreGet_Destroy

Parameters

new_get	the location where to store the pointer to the objStoreGet object.
obs	the pointer to the objStore object.
name	the name of the object to pull data from.
opts	the pointer to the objStoreOptions object, possibly NULL.

objStoreGet_Info()

NATS_EXTERN natsStatus objStoreGet_Info	(	const objStoreInfo **	new_info,
		objStoreGet *	get
	)

Allows the user to get information about the pulled object.

Warning

The objStoreInfo is owned by the objStoreGet object and MUST NOT be destroyed or altered by the user. In other words, do not call objStoreInfo_Destroy on the objStoreInfo pointer returned by this function. Instead, call objStoreGet_Destroy to free allocated memory.

Parameters

new_info	the location where to store the pointer to the objStoreInfo object.
get	the pointer to the objStoreGet object owning this information object.

objStoreGet_Read()

NATS_EXTERN natsStatus objStoreGet_Read	(	bool *	done,
		void **	new_data,
		int *	dataLen,
		objStoreGet *	get,
		int64_t	timeout
	)

This call returns up to a chunk of bytes of the object. The dataLen will be set with the number of bytes that were returned.

The done boolean will be set to true if the call detects that this was the last bit of data that consistuted this object. Calling this function again would result in the NATS_ILLEGAL_STATE error returned.

Warning

The returned data must be freed by the user.

See objStore_Get for example on how to use the set of APIs.

See also

objStore_Get

Parameters

done	the location where to store the boolean indicating if this is the last read.
new_data	the location where to store the pointer to a chunk of the object's data.
dataLen	the location where to store the number of bytes returned.
get	the pointer to the objStoreGet object.
timeout	the amount of time (in milliseconds) allowed to perform the operation.

objStoreGet_ReadAll()

NATS_EXTERN natsStatus objStoreGet_ReadAll	(	void **	new_data,
		int *	dataLen,
		objStoreGet *	get,
		int64_t	timeout
	)

This call returns all data representing the object, or the remaining of the data if some were already collected using objStoreGet_Read.

This function should be called only once (except in the case of a NATS_TIMEOUT), if not the error NATS_ILLEGAL_STATE error will be returned.

Warning

The returned data must be freed by the user.

See objStore_Get for example on how to use the set of APIs.

See also

objStore_Get

Parameters

new_data	the location where to store the pointer to a chunk of the object's data.
dataLen	the location where to store the number of bytes returned.
get	the pointer to the objStoreGet object.
timeout	the amount of time (in milliseconds) allowed to perform the operation.

objStoreGet_Destroy()

NATS_EXTERN void objStoreGet_Destroy

(

objStoreGet *

get

)

Releases memory allocated for this objStoreGet object.

Parameters

get	the pointer to the objStoreGet object.

objStore_GetString()

NATS_EXTERN natsStatus objStore_GetString	(	char **	new_str,
		objStore *	obs,
		const char *	name,
		objStoreOptions *	opts
	)

This is a convenience function to pull an object from this object store and return it as a string.

If the object does not exist, NATS_NOT_FOUND will be returned.

The option objStoreOptions.ShowDeleted can be provided to return an object even if it was marked as deleted.

Here is an example:

• objStoreOptions o;
   
  objStoreOptions_Init(&o);
  s = objStore_GetString(&str, obs, "myobject", &o);
  if (s == NATS_OK)
  {
       // Do something with the string
       ...
       // It is the user responsibility to free the memory.
       free(str);
  }
  ...

Note

The string is NULL terminated, therefore the allocated memory is one byte more than the size reported in the object's objStoreInfo.Size field.

Warning

The returned string must be freed when no longer needed to reclaim memory.

Parameters

new_str	the location where to store the pointer to the retrieved string.
obs	the pointer to the objStore object.
name	the name of the object to pull the string from.
opts	the pointer to the objStoreOptions object, possibly NULL.

objStore_GetBytes()

NATS_EXTERN natsStatus objStore_GetBytes	(	void **	new_data,
		int *	dataLen,
		objStore *	obs,
		const char *	name,
		objStoreOptions *	opts
	)

This is a convenience function to pull an object from this object store and return it as a byte array.

If the object does not exist, NATS_NOT_FOUND will be returned.

The option objStoreOptions.ShowDeleted can be provided to return an object even if it was marked as deleted. See objStore_GetString for an example on how to do so.

Warning

The returned byte array must be freed when no longer needed to reclaim memory.

Parameters

new_data	the location where to store the pointer to the retrieved byte array.
dataLen	the location where to store the number of bytes of the byte array.
obs	the pointer to the objStore object.
name	the name of the object to pull the string from.
opts	the pointer to the objStoreOptions object, possibly NULL.

objStore_GetFile()

NATS_EXTERN natsStatus objStore_GetFile	(	objStore *	obs,
		const char *	name,
		const char *	fileName,
		objStoreOptions *	opts
	)

This is a convenience function to pull an object from this object store and place it in a file. If the file already exists, it will be overwritten, otherwise it will be created.

If the object does not exist, NATS_NOT_FOUND will be returned.

The option objStoreOptions.ShowDeleted can be provided to return an object even if it was marked as deleted. See objStore_GetString for an example on how to do so.

Parameters

obs	the pointer to the objStore object.
name	the name of the object to pull the string from.
fileName	the name of the file to write the content into.
opts	the pointer to the objStoreOptions object, possibly NULL.