The mem package

Since version 1.5, C Python API define a buffer interface which allow an object to expose its internal data as a set of memory segments. This interface has been widely used by C coded extension but was mostly inaccessible from Python.

The mem package contain various tools to handle objects which define a single segment buffer interface: the object storage memory is contained into a contigous memory area.

Download mem 0.3 (24 Ko)

tools module

This module provide various low level functions to access objects raw memory, with the single-segment buffer API. It defines the following functions:

chunk module

The chunk module define a new object type which can be used to allocate raw segments of memory. Theses objects support the buffer API.

chunks objects are exposed as a mutable sequence of length one strings. They support all the methods required by the sequence protocol: len(), slicing, concatenation, repetition, iteration, indexing...

chunks objects support also the pickle protocol.

The following method are defined by chunk objects:

memslice module

This module define a new kind of objects which can be used to keep reference to a slice of raw memory from objects which define the buffer interface (strings, chunks, mmap ...). This object define itself the single-segment buffer API, so that it can be used instead of the target object on any operation which need an object with the buffer API.

The object define the following attributes:

The object also support the len(s) method.These method return the length of the accessible buffer. This is not necessairely equal to s.stop-s.length.

memslice object also handle slicing. This operation return a new memslice object.

memslice objects support also the pickle protocol, the base object must then support pickling itself.

This module is a rewrite of Python buffer() object which can be usable with relocatable and/or writable buffer segments. Another difference is that it use the common (start,stop) convention, instead of a more specific (start,length).

StrArray module

The module defines an array like adaptator to access a string or any object which define the buffer interface. The wrapped object must also define a .resize(n) method if the emulated sequence need to be able to change it's size. An array is a sequence of monotonic data values.

StrArray object support all the sequence protocol except the non-inplace form of the * and + operators. Note also that some other parts of the protocol are dependent of the wrapped objects: for sample you can't use mutable methods of the sequence protocol on a read-only buffer. Slicing behavior is also dependent of the wrapped object: you will get slice-by-copy on most buffer(str,chunks...) but some will provide slicing-by-view(memslice).

Array object add support for the non-inplace version of * and +.

This module is mostly a demo.

StrIO module

This class provide a file like object to access a string or any object which define the buffer interface. The wrapped object must also define a .resize(n) method if the emulated file need to be able to change it's size.

  • StrIO([data])

    Return a new StrIO object. data is the wrapped buffer, it default to a memory chunk.

  • StringIO([value])

    Create a new StrIO object which wrap a memory chunk. If value is provided it will be copied into the memory chunk.

This module is mostly a demo.