Robust C++: Object Pools
Greg Utas - 07/May/2020
Greg Utas - 07/May/2020
[SHOWTOGROUPS=4,20]
Recovering from memory leaks
When a system runs for a long time, using a shared heap can eventually result in a crash because of gradual fragmentation and memory leaks. By allocating objects from pools of fixed-size blocks, a system can limit fragmentation and use a background garbage collector to recover leaked blocks. This article presents an ObjectPool class that provides these capabilities.
A system that needs to be continuously available must recover from memory leaks so that it doesn't need to be periodically shut down for "routine maintenance". This article describes how object pools help to meet this requirement. Here, object pool does not refer to a pool of shared objects that are never destroyed. Rather, it refers to objects whose memory is allocated from a pool of fixed-size blocks instead of the heap.
Background
In C++, an object allocated from the heap, by the default implementation of operator new, must be explicitly deleted to return its memory to the heap. A memory leak results when deletion doesn't occur, usually because the pointer to the object gets lost. The addition of smart pointers in C++11 has reduced this risk, but it still exists (if a unique_ptr gets trampled, for example).
Another risk is memory fragmentation. Holes develop as memory blocks of differing sizes are allocated from, and returned to, the heap. Over a long period of time, this reduces the amount of effectively available memory unless the heap manager spends time merging adjacent free areas and implementing a best-fit policy.
The use of object pools allows a system to recover from memory leaks and avoid escalating fragmentation. As we shall see, it also enables some other useful capabilities.
Using the Code
Unlike many techniques, it is often possible to introduce object pools to a large legacy system without the need for significant reengineering. The reason is that allocating and freeing an object's memory is encapsulated by operators new and delete. By tweaking the class hierarchy, the usual versions of these operators, which use the default heap, are easily replaced with versions that use an object pool.
As in Robust C++: Safety Net, the code that we will look at is taken from the Robust Services Core (RSC), an open-source framework for building robust C++ applications. But this time, the code is more amenable to being copied into an existing project and modified to meet its needs.
Overview of the Classes
The classes are implemented in RSC's Для просмотра ссылки Войдиили Зарегистрируйся directory and follow the practice of being defined and implemented in a .h and .cpp of the same name. It should therefore be easy for you to find their full versions.
ObjectPool
ObjectPool is the base class for object pools, and each of its subclasses is a singleton that implements one pool. These subclasses do little other than invoke ObjectPool's constructor with the appropriate arguments. Each subclass, when created, gets added to an ObjectPoolRegistry that tracks all of the system's object pools. The blocks in each pool are allocated during system initialization so that the system can focus on its work once it is up and running.
Pooled
Pooled is the base class for objects whose memory comes from a pool. It overrides operator delete to return a block to its pool when an object is deleted. It also defines some data members that a pool uses to manage each block.
ObjectPoolAudit
ObjectPoolAudit is a thread which periodically wakes up to invoke a function that finds and returns orphaned blocks to the pool. Applications are still expected to delete objects, so the audit exists to fix memory leaks that could gradually cause the system to run out of memory. The audit uses a typical mark-and-sweep strategy but could be termed a background, rather than a foreground, garbage collector. It can run less frequently than a regular garbage collector, without freezing the system for an unbecoming length of time while performing its work.
Walkthroughs
The code in this article has been edited to remove things that would distract from the central concepts. These things are important in some applications, less so in others. If you look at the full version of the code, you will run across them, so a summary of what was removed is provided in Deleted Code.
Creating an Object Pool
Each singleton subclass of ObjectPool invokes its base class constructor to create its pool:
Memory::Align aligns each block to the underlying platform's word size (32 or 64 bits). Similar to a heap, an object pool needs some data to manage its block. It puts a BlockHeader at the start of each block:
The constructor initialized a queue (freeq_) for blocks that have yet to be allocated. RSC provides two queue templates, Q1Way and Q2Way. Their implementation differs from STL queues in that they never allocate memory after the system initializes. Instead, the class whose objects will be queued provides a ptrdiff_t offset to a data member that serves as a link to the next item in the queue. That offset was the argument to freeq_.Init().
A key design decision is the number of pools. The recommended approach is to use a common pool for all classes that derive from the same major framework class. RSC's NodeBase, for example, defines an object pool for the buffers used for inter-thread messaging. Note that the size of a pool's blocks must be somewhat larger than, for example, sizeof(MsgBuffer) so that subclasses will have room to add data of their own.
Using a common pool for all classes that derive from the same framework class significantly simplifies the engineering of pool sizes. Each pool must have enough blocks to handle times of peak load, when the system is maxed out. If every subclass had its own pool, each pool would need enough blocks to handle times when that subclass just happened to be especially popular. Having the subclasses share a pool smooths out such fluctuations, reducing the total number of blocks required.
Increasing the Size of an Object Pool
Whether a pool is creating its initial pool of blocks during system initialization, or whether it is allocating more blocks while in service, the code is the same:
Note that blocks are allocated in segments of 1K blocks each, such that an individual block is addressed by blocks_[j]. This makes it easy to allocate more blocks when the system is running: just add another segment. The fact that blocks in a segment are contiguous is useful for other reasons that will appear later.
[/SHOWTOGROUPS]
Recovering from memory leaks
When a system runs for a long time, using a shared heap can eventually result in a crash because of gradual fragmentation and memory leaks. By allocating objects from pools of fixed-size blocks, a system can limit fragmentation and use a background garbage collector to recover leaked blocks. This article presents an ObjectPool class that provides these capabilities.
- .../KB/cpp/5166096/master.zip
- ..../KB/cpp/5166096/master.zip
A system that needs to be continuously available must recover from memory leaks so that it doesn't need to be periodically shut down for "routine maintenance". This article describes how object pools help to meet this requirement. Here, object pool does not refer to a pool of shared objects that are never destroyed. Rather, it refers to objects whose memory is allocated from a pool of fixed-size blocks instead of the heap.
Background
In C++, an object allocated from the heap, by the default implementation of operator new, must be explicitly deleted to return its memory to the heap. A memory leak results when deletion doesn't occur, usually because the pointer to the object gets lost. The addition of smart pointers in C++11 has reduced this risk, but it still exists (if a unique_ptr gets trampled, for example).
Another risk is memory fragmentation. Holes develop as memory blocks of differing sizes are allocated from, and returned to, the heap. Over a long period of time, this reduces the amount of effectively available memory unless the heap manager spends time merging adjacent free areas and implementing a best-fit policy.
The use of object pools allows a system to recover from memory leaks and avoid escalating fragmentation. As we shall see, it also enables some other useful capabilities.
Using the Code
Unlike many techniques, it is often possible to introduce object pools to a large legacy system without the need for significant reengineering. The reason is that allocating and freeing an object's memory is encapsulated by operators new and delete. By tweaking the class hierarchy, the usual versions of these operators, which use the default heap, are easily replaced with versions that use an object pool.
As in Robust C++: Safety Net, the code that we will look at is taken from the Robust Services Core (RSC), an open-source framework for building robust C++ applications. But this time, the code is more amenable to being copied into an existing project and modified to meet its needs.
Overview of the Classes
The classes are implemented in RSC's Для просмотра ссылки Войди
ObjectPool
ObjectPool is the base class for object pools, and each of its subclasses is a singleton that implements one pool. These subclasses do little other than invoke ObjectPool's constructor with the appropriate arguments. Each subclass, when created, gets added to an ObjectPoolRegistry that tracks all of the system's object pools. The blocks in each pool are allocated during system initialization so that the system can focus on its work once it is up and running.
Pooled
Pooled is the base class for objects whose memory comes from a pool. It overrides operator delete to return a block to its pool when an object is deleted. It also defines some data members that a pool uses to manage each block.
ObjectPoolAudit
ObjectPoolAudit is a thread which periodically wakes up to invoke a function that finds and returns orphaned blocks to the pool. Applications are still expected to delete objects, so the audit exists to fix memory leaks that could gradually cause the system to run out of memory. The audit uses a typical mark-and-sweep strategy but could be termed a background, rather than a foreground, garbage collector. It can run less frequently than a regular garbage collector, without freezing the system for an unbecoming length of time while performing its work.
Walkthroughs
The code in this article has been edited to remove things that would distract from the central concepts. These things are important in some applications, less so in others. If you look at the full version of the code, you will run across them, so a summary of what was removed is provided in Deleted Code.
Creating an Object Pool
Each singleton subclass of ObjectPool invokes its base class constructor to create its pool:
Код:
ObjectPool::ObjectPool(ObjectPoolId pid, size_t size, size_t segs) :
blockSize_(0),
segIncr_(0),
segSize_(0),
currSegments_(0),
targSegments_(segs),
corruptQHead_(false)
{
// The block size must account for the header above each Pooled object.
//
blockSize_ = BlockHeaderSize + Memory::Align(size);
segIncr_ = blockSize_ >> BYTES_PER_WORD_LOG2;
segSize_ = segIncr_ * ObjectsPerSegment;
for(auto i = 0; i < MaxSegments; ++i) blocks_[i] = nullptr;
// Initialize the pool's free queue of blocks.
//
freeq_.Init(Pooled::LinkDiff());
// Set the pool's identifier and add it to the registry of object pools.
//
pid_.SetId(pid);
Singleton< ObjectPoolRegistry >::Instance()->BindPool(*this);
}Memory::Align aligns each block to the underlying platform's word size (32 or 64 bits). Similar to a heap, an object pool needs some data to manage its block. It puts a BlockHeader at the start of each block:
Код:
// The header for a Pooled (a block in the pool). Data in the header
// survives when an object is deleted.
//
struct BlockHeader
{
ObjectPoolId pid : 8; // the pool to which the block belongs
PooledObjectSeqNo seq : 8; // the block's incarnation number
};
const size_t BlockHeader::Size = Memory::Align(sizeof(BlockHeader));
// This struct describes the top of an object block for a class that
// derives from Pooled.
//
struct ObjectBlock
{
BlockHeader header; // block management information
Pooled obj; // the actual location of the object
};
constexpr size_t BlockHeaderSize = sizeof(ObjectBlock) - sizeof(Pooled);
For reference, here are the members of Pooled that are relevant to this article:
Hide Shrink [IMG]https://www.codeproject.com/images/arrow-up-16.png[/IMG] Copy Code
// A pooled object is allocated from an ObjectPool created during system
// initialization rather than from the heap.
//
class Pooled : public Object
{
friend class ObjectPool;
public:
// Virtual to allow subclassing.
//
virtual ~Pooled() = default;
// Returns the offset to link_.
//
static ptrdiff_t LinkDiff();
// Overridden to claim blocks that this object owns.
//
void ClaimBlocks() override;
// Clears the object's orphaned_ field so that the object pool audit
// will not reclaim it. May be overridden, but the base class version
// must be invoked.
//
void Claim() override;
// Overridden to return a block to its object pool.
//
static void operator delete(void* addr);
// Deleted to prohibit array allocation.
//
static void* operator new[](size_t size) = delete;
protected:
// Protected because this class is virtual.
//
Pooled();
private:
// Link for queueing the object.
//
Q1Link link_;
// True if allocated for an object; false if on free queue.
//
bool assigned_;
// Zero for a block that is in use. Incremented each time through the
// audit; if it reaches a threshold, the block is deemed to be orphaned
// and is recovered.
//
uint8_t orphaned_;
// Used by audits to avoid invoking functions on a corrupt block. The
// audit sets this flag before it invokes any function on the object.
// If the object's function traps, the flag is still set when the audit
// resumes execution, so it knows that the block is corrupt and simply
// recovers it instead of invoking its function again. If the function
// returns successfully, the audit immediately clears the flag.
//
bool corrupt_;
// Used to avoid double logging.
//
bool logged_;
};The constructor initialized a queue (freeq_) for blocks that have yet to be allocated. RSC provides two queue templates, Q1Way and Q2Way. Their implementation differs from STL queues in that they never allocate memory after the system initializes. Instead, the class whose objects will be queued provides a ptrdiff_t offset to a data member that serves as a link to the next item in the queue. That offset was the argument to freeq_.Init().
A key design decision is the number of pools. The recommended approach is to use a common pool for all classes that derive from the same major framework class. RSC's NodeBase, for example, defines an object pool for the buffers used for inter-thread messaging. Note that the size of a pool's blocks must be somewhat larger than, for example, sizeof(MsgBuffer) so that subclasses will have room to add data of their own.
Using a common pool for all classes that derive from the same framework class significantly simplifies the engineering of pool sizes. Each pool must have enough blocks to handle times of peak load, when the system is maxed out. If every subclass had its own pool, each pool would need enough blocks to handle times when that subclass just happened to be especially popular. Having the subclasses share a pool smooths out such fluctuations, reducing the total number of blocks required.
Increasing the Size of an Object Pool
Whether a pool is creating its initial pool of blocks during system initialization, or whether it is allocating more blocks while in service, the code is the same:
Код:
bool ObjectPool::AllocBlocks()
{
auto pid = Pid();
while(currSegments_ < targSegments_)
{
// Allocate memory for the next group of blocks.
//
auto size = sizeof(uword) * segSize_;
blocks_[currSegments_] = (uword*) Memory::Alloc(size, false);
if(blocks_[currSegments_] == nullptr) return false;
++currSegments_;
totalCount_ = currSegments_ * ObjectsPerSegment;
// Initialize each block and add it to the free queue.
//
auto seg = blocks_[currSegments_ - 1];
for(size_t j = 0; j < segSize_; j += segIncr_)
{
auto b = (ObjectBlock*) &seg[j];
b->header.pid = pid;
b->header.seq = 0;
b->obj.link_.next = nullptr;
b->obj.assigned_ = false;
b->obj.orphaned_ = OrphanThreshold;
EnqBlock(&b->obj);
}
}
return true;
}Note that blocks are allocated in segments of 1K blocks each, such that an individual block is addressed by blocks_[j]. This makes it easy to allocate more blocks when the system is running: just add another segment. The fact that blocks in a segment are contiguous is useful for other reasons that will appear later.
[/SHOWTOGROUPS]
Последнее редактирование:
