API Documentation

All modules listed below are under the “nvm” module.

pmem – low level persistent memory support

class nvm.pmem.DrainContext(memory_buffer, unmap=True)

A context manager that will automatically drain the specified memory buffer.

Parameters:memory_buffer – the MemoryBuffer object.
class nvm.pmem.FlushContext(memory_buffer, unmap=True)

A context manager that will automatically flush the specified memory buffer.

Parameters:memory_buffer – the MemoryBuffer object.
class nvm.pmem.MemoryBuffer(buffer_)

A file-like I/O (similar to cStringIO) for persistent mmap’d regions.

read(size=0)

Read data from the buffer.

Parameters:size – size to read, zero equals to entire buffer size.
Returns:data read.
seek(pos)

Moves the cursor position in the buffer.

Parameters:pos – the new cursor position
write(data)

Write data into the buffer.

Parameters:data – data to write into the buffer.
nvm.pmem.check_version(major_required, minor_required)

Checks the libpmem version according to the specified major and minor versions required.

Parameters:
  • major_required – Major version required.
  • minor_required – Minor version required.
Returns:

returns True if the nvm has the required version, or raises a RuntimeError exception in case of failure.

nvm.pmem.drain(memory_buffer)

Wait for any PM stores to drain from HW buffers.

Parameters:memory_buffer – the MemoryBuffer object.
nvm.pmem.flush(memory_buffer)

Flush processor cache for the given memory region.

Parameters:memory_buffer – the MemoryBuffer object.
nvm.pmem.has_hw_drain()

This function returns true if the machine supports the hardware drain function for persistent memory, such as that provided by the PCOMMIT instruction on Intel processors.

Returns:return True if it has hardware drain, False otherwise.
nvm.pmem.is_pmem(memory_buffer)

Return true if entire range is persistent memory.

Returns:True if the entire range is persistent memory, False otherwise.
nvm.pmem.map(file_, size)

Map the entire file for read/write access

Parameters:file – The file descriptor of a file object.
Returns:The mapping, an exception will rise in case of error.
nvm.pmem.msync(memory_buffer)

Flush to persistence via msync().

Parameters:memory_buffer – the MemoryBuffer object.
Returns:the msync() return result, in case of msync() error, an exception will rise.
nvm.pmem.persist(memory_buffer)

Make any cached changes to a range of pmem persistent.

Parameters:memory_buffer – the MemoryBuffer object.
nvm.pmem.unmap(memory_buffer)

Unmap the specified region.

Parameters:memory_buffer – the MemoryBuffer object.

pmemlog – pmem-resident log file

class nvm.pmemlog.LogPool(log_pool)

This class represents the Log Pool opened or created using create() or open().

append(buf)

This method appends from buffer to the current write offset in the log memory pool plp. Calling this function is analogous to appending to a file. The append is atomic and cannot be torn by a program failure or system crash.

On success, zero is returned. On error, -1 is returned and errno is set.

close()

This method closes the memory pool. The log memory pool itself lives on in the file that contains it and may be re-opened at a later time using open().

nbyte()

This method returns the amount of usable space in the log pool. This method may be used to determine how much usable space is available after libpmemlog has added its metadata to the memory pool.

Note

You can also use len() to get the usable space.

Returns:amount of usable space in the log pool.
rewind()

This method resets the current write point for the log to zero. After this call, the next append adds to the beginning of the log.

tell()

This method returns the current write point for the log, expressed as a byte offset into the usable log space in the memory pool. This offset starts off as zero on a newly-created log, and is incremented by each successful append operation. This function can be used to determine how much data is currently in the log.

Returns:the current write point for the log, expressed as a byte offset.
walk(func, chunk_size=0)

This function walks through the log pool, from beginning to end, calling the callback function for each chunksize block of data found. The chunksize argument is useful for logs with fixed-length records and may be specified as 0 to cause a single call to the callback with the entire log contents.

Parameters:
  • chunk_size – chunk size or 0 for total length (default to 0).
  • func – the callback function, should return 1 if it should continue walking through the log, or 0 to terminate the walk.
nvm.pmemlog.check(filename)

This method performs a consistency check of the file indicated and returns True if the memory pool is found to be consistent. Any inconsistencies found will cause this function to return False, in which case the use of the file with libpmemlog will result in undefined behavior.

Returns:True if memory pool is consistent, False otherwise.
nvm.pmemlog.check_version(major_required, minor_required)

Checks the libpmemlog version according to the specified major and minor versions required.

Parameters:
  • major_required – Major version required.
  • minor_required – Minor version required.
Returns:

returns True if the nvm has the required version, or raises a RuntimeError exception in case of failure.

nvm.pmemlog.create(filename, pool_size=2097152, mode=438)

The create() function creates a log memory pool with the given total pool_size. Since the transactional nature of a log memory pool requires some space overhead in the memory pool, the resulting available log size is less than poolsize, and is made available to the caller via the nbyte() function.

Note

If the error prevents any of the pool set files from being created, this function will raise an exception.

Parameters:
  • filename – specifies the name of the memory pool file to be created.
  • pool_size – the size of the pool (default to 2MB).
  • mode – specifies the permissions to use when creating the file.
Returns:

the new log memory pool created.

Return type:

LogPool

nvm.pmemlog.open(filename)

This function opens an existing log memory pool, returning a memory pool.

Note

If an error prevents the pool from being opened, this function will rise an exception.

Parameters:filename – Filename must be an existing file containing a log memory pool as created by the create() method. The application must have permission to open the file and memory map it with read/write permissions.
Returns:the log memory pool.
Return type:LogPool

pmemblk – arrays of pmem-resident blocks

class nvm.pmemblk.BlockPool(block_pool)

This class represents the Block Pool opened or created using create() or open().

bsize()

This method returns the block size of the specified block memory pool. It’s the value which was passed as block size to create().

Returns:the block size.
close()

This method closes the memory pool. The block memory pool itself lives on in the file that contains it and may be re-opened at a later time using open().

nblock()

This method returns the usable space in the block memory pool, expressed as the number of blocks available.

Returns:usable space in block memory pool in number of blocks.
read(block_num)

This method reads a block from memory pool at specified block number.

Note

Reading a block that has never been written will return an empty buffer.

Returns:data at block.
set_error(block_num)

This method sets the error state for block number blockno in memory pool. A block in the error state returns errno EIO when read. Writing the block clears the error state and returns the block to normal use.

Returns:On success, zero is returned. On error, an exception will be raised.
set_zero(block_num)

This method writes zeros to block number blockno in memory pool. Using this function is faster than actually writing a block of zeros since libpmemblk uses metadata to indicate the block should read back as zero.

Returns:On success, zero is returned. On error, an exception will be raised.
write(data, block_num)

This method writes a block from data to block number blockno in the memory pool. The write is atomic with respect to other reads and writes. In addition, the write cannot be torn by program failure or system crash; on recovery the block is guaranteed to contain either the old data or the new data, never a mixture of both.

Returns:On success, zero is returned. On error, an exception will be raised.
nvm.pmemblk.check(filename, block_size=0)

This function performs a consistency check of the file indicated by path and returns True if the memory pool is found to be consistent. Any inconsistencies found will cause it to return False, in which case the use of the file with libpmemblk will result in undefined behavior.

Note

When block size is non-zero, it will compare it to the block size of the pool and return False when they don’t match.

Returns:True if memory pool is consistent, False otherwise.
nvm.pmemblk.check_version(major_required, minor_required)

Checks the libpmemblk version according to the specified major and minor versions required.

Parameters:
  • major_required – Major version required.
  • minor_required – Minor version required.
Returns:

returns True if the nvm has the required version, or raises a RuntimeError exception in case of failure.

nvm.pmemblk.create(filename, block_size, pool_size=2097152, mode=438)

This function function creates a block memory pool with the given total pool size divided up into as many elements of block size as will fit in the pool.

Note

Since the transactional nature of a block memory pool requires some space overhead in the memory pool, the resulting number of available blocks is less than poolsize / block size, and is made available to the caller via the nblock().

If the error prevents any of the pool set files from being created, this function will raise an exception.

Parameters:
  • filename – specifies the name of the memory pool file to be created.
  • block_size – the size of the blocks.
  • pool_size – the size of the pool (default to 2MB).
  • mode – specifies the permissions to use when creating the file.
Returns:

the new block memory pool created.

Return type:

BlockPool

nvm.pmemblk.open(filename, block_size=0)

This function opens an existing block memory pool, returning a memory pool.

Note

If an error prevents the pool from being opened, this function will rise an exception. If the block size provided is non-zero, it will verify the given block size matches the block size used when the pool was created. Otherwise, it will open the pool without verification of the block size.

Parameters:filename – Filename must be an existing file containing a block memory pool as created by the create() method. The application must have permission to open the file and memory map it with read/write permissions.
Returns:the block memory pool.
Return type:BlockPool