pmemobj
— Pesistent Python objects¶
Creating and Accessing a PersistentObjectPool
¶
-
pmemobj.
create
(filename, pool_size=MIN_POOL_SIZE, mode=0o666, debug=False)¶ Return a
PersistentObjectPool
backed by a file named filename, allocating pool_size bytes for the pool, and setting the mode of the file on the filesystem to mode. Raise anOSError
if the file already exists. Pass debug to thePersistentObjectPool
constructor.If filename is in a filesystem backed by persistent memory, the memory will be directly accessed. Otherwise persistent memory will be emulated by memory mapping a disk file.
The actual amount of memory available for objects is smaller than pool_size transaction and object management overhead. The default is the default used by
libpmemobj
.
-
pmemobj.
open
(filename, debug=False)¶ Return a
PersistentObjectPool
backed by the file named filename. Raise an anOSError
if the file does not exist. If the previous shutdown was not clean, call thePersistentObjectPool.gc
method. Pass debug to thePersistentObjectPool
constructor.
-
class
pmemobj.
PersistentObjectPool
(filename, flag='w', pool_size=MIN_POOL_SIZE, mode=0x666, debug=False)¶ Open or create a persistent object pool backd by filename. If flag is
w
, raise anOSError
if the file does not exist and otherwise open it for reading and writing. If flag isx
, raise anOSError
if the file already exists, and otherwise create the file and open it for reading and writing. If flag isc
, create the file if it does not exist, but in any case open it for reading and writing.If the file gets created, allocate pool_size bytes for the pool, and set its mode in the filesystem to mode.
If the object pool was previously not closed cleanly, call
gc()
.Use debug as the default value for the debug parameter to the
gc()
method.-
root
¶ The “root” object of the pool. This can be set to any object that can be persisted, but it is really only useful to set it to a Perisistent collection type. Only objects that are reachable by traversing the object graph starting from the root object will be preserved once the object pool is closed.
-
gc
(debug=None)¶ Free all unreferenced objects: objects not accessible by tracing the object graph starting at the
root
object.
-
new
(typ, *args, **kw)¶ Create a new instance of typ managed by this pool, passing its constructor args and kw. typ must support the
Persistent
API.
-
persist_via_pickle
(*types)¶ Add types to the list of types that will be persisted via pickle. Nominated types must be non-container immutable types (this is not currently enforced, but confusing things will happen if you violate it). If a version of pmemobj to which support for a given type has been added is used to open a pool with instances of that type stored via pickle, the object will be resurrected from pickle, but any new instances written to the pool will use the direct support.
-
transaction
()¶ Return a context manager object that manages a transaction. If the context is exited normally, all changes to objects managed by the pool should be committed; if the context exits abnormally or the program stops running for any reason in the middle of the context, then none of the changes to the persistent objects inside the transaction context should be visible. Note that the transaction does not affect changes to normal Python objects; only changes to Persistent objects will be rolled back on abnormal exit.
-
Managing Persistent Memory¶
-
class
pmemobj.
MemoryManager
(pool_pr, type_table=None)¶ Create a manager for a
PersistentObjectPool
’s memory. This class should never be instantiated directly, but instead the automatically created instance should be accessed through a pool object.All of the methods below are atomic from the point of view of the caller. If the program crashes in the middle of the method it will either have completed or on pool reopen it will be as if it had never been started. All methods may be called from inside a transaction to make them part of a larger atomic unit of change.
-
new
(typ, *args, **kw)¶ Create a new instance of typ managed by the pool, passing its constructor args and kw. typ must support the
Persistent
API.
-
transaction
()¶ Return a context manager object that manages a transaction. If the context is exited normally, all changes to objects managed by the pool should be committed; if the context exits abnormally or the program stops running for any reason in the middle of the context, then none of the changes to the persistent objects inside the transaction context should be visible when the pool is next opened.
-
otuple
(oid)¶ Ensure that oid is in tuple form. An
oid
retreived from memory is actually a pointer to the memory the oid was retrieved from, so if contents of that memory location changes the value of the rawoid
would change. This method copies theoid
data into a tuple not subject to such modification, but which can be assigned to a memory field to store its value at that location.All
MemoryManager
methods that return oids return them in tuple form.
-
alloc
(size, type_num=POBJECT_TYPE_NUM)¶ Return an
oid
pointing to size bytes of newly allocated persistent memory, passing type_num to libpmemobj as the new memory object’s type. Raise an error if called outside of anytransaction()
.A
Persistent
class should use POBJECT_TYPE_NUM for its base memory allocation, but should use a unique number for any non-PObject memory structures it allocates. (There is currently no way to manage allocating these numbers to guarnatee uniqueness, but in fact as long as something other than POBJECT_TYPE_NUM is used, nothing should break even if the number collides with at used by a differentPersistent
type, you just lose some memory type safety.)
-
zalloc
(size, type_num=POBJECT_TYPE_NUM)¶ Same as
alloc()
, but the allocated persistent memory is also zeroed.
-
free
(oid)¶ Return the persistent memory pointed to by oid to the pool, so that it is avaiable for future allocation. Raise an error if called outside of any transaction.
-
realloc
(oid, size, type_num=None)¶ Return an
oid
pointing to size bytes of newly allocated persistent memory and copy the data pointed to by oid into it, truncating or zero-filling as needed. Raise an error if type_num is notNone
and does not match the pmem type of oid.free()
the memory originally pointed to by oid. Raise an error if called outside of any transaction.
-
zrealloc
(size, type_num=POBJECT_TYPE_NUM)¶ Same as
realloc()
, but the newly allocated persistent memory is also zeroed.
-
incref
(oid)¶ Increment the reference count of the
PObject
pointed to by oid.
-
decref
(oid)¶ Decrement the reference count of the
POjbect
pointed to by oid. If the reference count is zero after the decrement, then if the object has a_p_deallocate()
method call it, and in any case callfree()
on oid.
-
xdecref
(oid)¶ Call
decref()
on oid if oid is notOID_NULL
.decref()
should be used whever possible, so that cases where anoid
is unexpectedly null raise an error. If, however, the poitner can legitimately be null, this method eliminates the need for an if test, and this is a common enough case to be worth having extra method.
-
persist
(obj)¶ Return an
oid
pointing to the representation of obj in peristent memory, creating that representation if necessary. obj must be one of the directly supported immutable types, or one of the immutable types nominated for persistence viapickle
, or aPersistent
type.
-
resurrect
(oid)¶ Return a Python object representing the
POjbect
stored at oid. This may be a pure Python object if the stored object is a non-container immutable, or is otherwise an object that redirects data accesses to data stored in persistent memory.
-
direct
(oid)¶ Return the real memory address of the persistent memory pointed to by oid.
-
Persistent Classes¶
-
class
pmemobj.
Persistent
¶ Persistent
is an Abstract Base Class for objects that implement thePersistent
interface. All classes that want to store their data in persistent memory and manage it must implement the interface described here, but they are not required to use the ABC as their base.-
_p_mm
¶ A
MemoryManager
instance from thePersistentObjectPool
in which this object is stored.
-
_p_oid
¶ The
oid
that points to thePObject
data structure in persistent memory that anchors this objects data.
-
_p_new
(manager)¶ Initialize the objects data structures when the object is initially created, and store the provided
MemoryManager
manager in_p_mm
and theoid
pointing to the initialized data structures (aPOjbect
) in_p_oid
.
-
_p_resurrect
(manager, oid)¶ Restore the object’s state from the data located at oid, using manager, storing the manager in
_p_mm
and the oid in_p_oid
.
-
_p_traverse
()¶ Return an iterable over the
oids
of all of the objects pointed to by this object.
-
_p_substructures
()¶ Return an iterable over the oids of all of the non-
PObject
data structures allocated by this object.
-
-
class
pmemobj.
PersistentList
([iterable])¶ A
Persistent
version of the normal Pythonlist
. Its behavior should be identical except for being persistent. (Note: currently slices are not supported.)
-
class
pmemobj.
PersistentDict
([mapping_or_iterable, ]**kwarg)¶ A
Persistent
version of the normal Pythondict
. Its behavior should be identical except for being persistent.
-
class
pmemobj.
PersistentObject
¶ Base class for user defined
Persistent
objects. May not be mixed with any otherPersistent
type.As with a normal class,
__init__
is called when the object is initially created. It is not called during object resurrection.-
_v__init__
()¶
This method is called both when the object is initially created and when the object is resurrected. It does nothing by default, but can be overridden to (re)acquire volatile resources. It is called before
__init__
during object creation.-