Robust C++: Initialization and Restarts
Greg Utas - 04/May/2020
Greg Utas - 04/May/2020
[SHOWTOGROUPS=4,20]
Structuring main() and quickly recovering from memory corruption
In a large system, main() can easily become a mess as different developers add their initialization code. This article presents a Module class that allows a system to be initialized in a structured, layered manner. It then evolves the design to show how the system can perform a quick restart, rather than a reboot, to recover from serious errors such as trampled memory.
In many C++ programs, the main function #includes the world and utterly lacks structure. This article describes how to initialize a system in a structured manner. It then discusses how to evolve the design to support recovery from serious errors (usually corrupted memory) by quickly reinitializing a subset of the system instead of having to reboot its executable.
Using the Code
The code in this article is taken from the Robust Services Core (RSC), a large repository that provides a framework for developing robust C++ applications. RSC's software is organized into static libraries, each in its own namespace. Much of the code excerpted in this article comes from the namespace NodeBase in the Для просмотра ссылки Войдиили Зарегистрируйся directory. NodeBase contains about 50K lines of code that provide base classes for things such as:
If you don't want to use RSC, you can copy and modify its source code to meet your needs, subject to the terms of its GPL-3.0 license.
RSC contains many details that are not relevant to this article, so the code that we look at will be excerpted from the relevant classes and functions, but with irrelevant details removed. Many of these details are nonetheless important and need to be considered if your approach is to copy and modify RSC software.
In most cases, RSC defines each class in a .h of the same name and implements it in a .cpp of the same name. You should therefore be able to easily find the full version of each class.
Initializing the System
We'll start by looking at how RSC initializes when the system boots up.
Module
Each Module subclass represents a set of interrelated source code files that provides some logical capability. Each of these subclasses is responsible for:
A module specifies its dependencies in its constructor and initializes its static library in its Startup function. Here is the outline of a typical module:
If each module's constructor instantiates the modules on which it depends, how are leaf modules created? The answer is that main creates them. The code for main will appear soon.
ModuleRegistry
The singleton ModuleRegistry appeared in the last line of the above constructor. It contains all of the system's modules, sorted by their dependencies (a partial ordering). ModuleRegistry also has a Startup function that initializes the system by invoking Startup on each module.
Thread, RootThread, and InitThread
In RSC, each thread derives from the base class Thread, which encapsulates a native thread and provides a variety of functions related to things like exception handling, scheduling, and inter-thread communication.
The first thread that RSC creates is RootThread, which wraps the native thread that the C++ run-time system created to run main. RootThread simply brings the system up to the point where it can create the next thread. That thread, InitThread, is responsible for initializing most of the system. Once initialization is complete, InitThread acts as a watchdog to ensure that threads are being scheduled, and RootThread acts as a watchdog to ensure that InitThread is running.
main()
After it echoes and saves any command line arguments, main simply instantiates leaf modules. RSC currently has 15 static libraries and, therefore, 15 modules. Modules that are instantiated transitively are commented out:
Once the system has initialized, entering the >modules command on the CLI displays the following, which is the order in which the modules were invoked to initialize their static libraries:
If an application built on RSC does not require a particular static library, the instantiation of its module can be commented out, and the linker will exclude all of that library's code from the executable.
main is the only code implemented outside a static library. It resides in the rsc directory, whose only source code file is main.cpp. All other software, whether part of the framework or an application, resides in a static library.
RootThread::Main
The last thing that main did was invoke RootThread::Main, which is a static function because RootThread has not yet been instantiated. Its job is to create the things that are needed to actually instantiate RootThread:
Invoking Thread::EnterThread leads to the invocation of RootThread::Enter, which implements RootThread's thread loop. RootThread::Enter creates InitThread, whose first task is to finish initializing the system. RootThread then goes to sleep, running a watchdog timer that is cancelled when InitThread interrupts RootThread to tell it that the system has been initialized. If the timer expires, the system failed to initialize: it is embarrassingly dead on arrival, so RootThread exits.
ModuleRegistry::Startup
To finish initializing the system, InitThread invokes ModuleRegistry::Startup. This function invokes each module's Startup function. It also records how long it took to initialize each module, code that has been deleted for clarity:
Once this function is finished, something very similar to this will have appeared on the console:
[/SHOWTOGROUPS]
Structuring main() and quickly recovering from memory corruption
In a large system, main() can easily become a mess as different developers add their initialization code. This article presents a Module class that allows a system to be initialized in a structured, layered manner. It then evolves the design to show how the system can perform a quick restart, rather than a reboot, to recover from serious errors such as trampled memory.
- .../KB/cpp/5254138/master.zip
- .../KB/cpp/5254138/master.zip
In many C++ programs, the main function #includes the world and utterly lacks structure. This article describes how to initialize a system in a structured manner. It then discusses how to evolve the design to support recovery from serious errors (usually corrupted memory) by quickly reinitializing a subset of the system instead of having to reboot its executable.
Using the Code
The code in this article is taken from the Robust Services Core (RSC), a large repository that provides a framework for developing robust C++ applications. RSC's software is organized into static libraries, each in its own namespace. Much of the code excerpted in this article comes from the namespace NodeBase in the Для просмотра ссылки Войди
- system initialization/reinitialization (the focus of this article)
- configuration parameters
- multi-threading
- object pooling
- CLI commands
- logging
- debugging tools
If you don't want to use RSC, you can copy and modify its source code to meet your needs, subject to the terms of its GPL-3.0 license.
RSC contains many details that are not relevant to this article, so the code that we look at will be excerpted from the relevant classes and functions, but with irrelevant details removed. Many of these details are nonetheless important and need to be considered if your approach is to copy and modify RSC software.
In most cases, RSC defines each class in a .h of the same name and implements it in a .cpp of the same name. You should therefore be able to easily find the full version of each class.
Initializing the System
We'll start by looking at how RSC initializes when the system boots up.
Module
Each Module subclass represents a set of interrelated source code files that provides some logical capability. Each of these subclasses is responsible for:
- specifying the other modules on which it depends
- initializing the set of source code files that it represents when the executable is launched
A module specifies its dependencies in its constructor and initializes its static library in its Startup function. Here is the outline of a typical module:
Код:
class SomeModule : public Module
{
friend class Singleton< SomeModule >;
private:
SomeModule() : Module()
{
// Modules 1 to N are the ones on which this module depends.
// Creating their singletons ensures that they will exist in
// the module registry when the system initializes. Because
// each module creates the modules on which it depends before
// it adds itself to the registry, the registry will contain
// modules in the (partial) order of their dependencies.
//
Singleton< Module1 >::Instance();
// ...
Singleton< ModuleN >::Instance();
Singleton< ModuleRegistry >::Instance()->BindModule(*this);
}
~SomeModule() = default;
void Startup() override; // details are specific to each module
};If each module's constructor instantiates the modules on which it depends, how are leaf modules created? The answer is that main creates them. The code for main will appear soon.
ModuleRegistry
The singleton ModuleRegistry appeared in the last line of the above constructor. It contains all of the system's modules, sorted by their dependencies (a partial ordering). ModuleRegistry also has a Startup function that initializes the system by invoking Startup on each module.
Thread, RootThread, and InitThread
In RSC, each thread derives from the base class Thread, which encapsulates a native thread and provides a variety of functions related to things like exception handling, scheduling, and inter-thread communication.
The first thread that RSC creates is RootThread, which wraps the native thread that the C++ run-time system created to run main. RootThread simply brings the system up to the point where it can create the next thread. That thread, InitThread, is responsible for initializing most of the system. Once initialization is complete, InitThread acts as a watchdog to ensure that threads are being scheduled, and RootThread acts as a watchdog to ensure that InitThread is running.
main()
After it echoes and saves any command line arguments, main simply instantiates leaf modules. RSC currently has 15 static libraries and, therefore, 15 modules. Modules that are instantiated transitively are commented out:
Код:
main_t main(int argc, char* argv[])
{
// Echo and save the arguments. MainArgs is a simple class
// that saves and provides access to the arguments.
//
std::cout << "ENTERING main(int argc, char* argv[])" << CRLF;
std::cout << " argc: " << argc << CRLF;
for(auto i = 0; i < argc; ++i)
{
string arg(argv[i]);
MainArgs::PushBack(arg);
std::cout << " argv[" << i << "]: " << arg << CRLF;
}
std::cout << std::flush;
// Instantiate the desired modules.
//
// Singleton< NbModule >::Instance();
// Singleton< NtModule >::Instance();
Singleton< CtModule >::Instance();
// Singleton< NwModule >::Instance();
// Singleton< SbModule >::Instance();
// Singleton< StModule >::Instance();
// Singleton< MbModule >::Instance();
// Singleton< CbModule >::Instance();
// Singleton< PbModule >::Instance();
Singleton< OnModule >::Instance();
Singleton< CnModule >::Instance();
Singleton< RnModule >::Instance();
Singleton< SnModule >::Instance();
Singleton< AnModule >::Instance();
// Singleton< DipModule >::Instance(); // usually omitted
return RootThread::Main();
}Once the system has initialized, entering the >modules command on the CLI displays the following, which is the order in which the modules were invoked to initialize their static libraries:
Код:
nb>modules
NodeBase.ModuleRegistry
this : 003B0660
// stuff deleted
modules [ModuleId]
size : 14
// stuff deleted
registry : 003B06A0
[1]: 003B0640 NodeBase.NbModule
[2]: 003B0E88 NodeTools.NtModule
[3]: 003B0620 CodeTools.CtModule
[4]: 003B0F08 NetworkBase.NwModule
[5]: 003B0EE8 SessionBase.SbModule
[6]: 003B0EC8 ControlNode.CnModule
[7]: 003B0F68 SessionTools.StModule
[8]: 003B0F88 MediaBase.MbModule
[9]: 003B0F48 CallBase.CbModule
[10]: 003B0F28 PotsBase.PbModule
[11]: 003B0EA8 OperationsNode.OnModule
[12]: 003B0FA8 RoutingNode.RnModule
[13]: 003B0FC8 ServiceNode.SnModule
[14]: 003B0FF0 AccessNode.AnModuleIf an application built on RSC does not require a particular static library, the instantiation of its module can be commented out, and the linker will exclude all of that library's code from the executable.
main is the only code implemented outside a static library. It resides in the rsc directory, whose only source code file is main.cpp. All other software, whether part of the framework or an application, resides in a static library.
RootThread::Main
The last thing that main did was invoke RootThread::Main, which is a static function because RootThread has not yet been instantiated. Its job is to create the things that are needed to actually instantiate RootThread:
Код:
main_t RootThread::Main()
{
// This loop is hypothetical because our Enter function (invoked
// through Thread::EnterThread and Thread::Start) never returns.
//
while(true)
{
// Load symbol information.
//
SysThreadStack::Startup(RestartReboot);
// Create the POSIX signals. They are needed now so that
// RootThread can register for signals when it is wrapped.
//
CreatePosixSignals();
// Wrap the root thread and enter it.
//
auto root = Singleton< RootThread >::Instance();
Thread::EnterThread(root);
}
}Invoking Thread::EnterThread leads to the invocation of RootThread::Enter, which implements RootThread's thread loop. RootThread::Enter creates InitThread, whose first task is to finish initializing the system. RootThread then goes to sleep, running a watchdog timer that is cancelled when InitThread interrupts RootThread to tell it that the system has been initialized. If the timer expires, the system failed to initialize: it is embarrassingly dead on arrival, so RootThread exits.
ModuleRegistry::Startup
To finish initializing the system, InitThread invokes ModuleRegistry::Startup. This function invokes each module's Startup function. It also records how long it took to initialize each module, code that has been deleted for clarity:
Код:
void ModuleRegistry::Startup()
{
for(auto m = modules_.First(); m != nullptr; modules_.Next(m))
{
m->Startup();
}
}Once this function is finished, something very similar to this will have appeared on the console:
Код:
ENTERING main(int argc, char* argv[])
argc: 1
argv[0]: C:\Users\gregu\Documents\rsc\rsc\Debug\rsc.exe
MODULE INITIALIZATION msecs invoked at
pre-Module.Startup 433 08:28:00.212
NodeBase.NbModule... 08:28:00.645
...initialized 62
NodeTools.NtModule... 08:28:00.717
...initialized 18
CodeTools.CtModule... 08:28:00.743
...initialized 18
NetworkBase.NwModule... 08:28:00.770
NET500 2-Aug-2019 08:28:00.785 on Reigi {1}
...initialized 121
SessionBase.SbModule... 08:28:00.900
...initialized 90
SessionTools.StModule... 08:28:01.001
...initialized 12
MediaBase.MbModule... 08:28:01.024
...initialized 13
CallBase.CbModule... 08:28:01.047
...initialized 20
PotsBase.PbModule... 08:28:01.078
...initialized 17
OperationsNode.OnModule... 08:28:01.106
...initialized 10
ControlNode.CnModule... 08:28:01.127
...initialized 11
RoutingNode.RnModule... 08:28:01.149
...initialized 11
ServiceNode.SnModule... 08:28:01.170
...initialized 35
AccessNode.AnModule... 08:28:01.218
...initialized 16
-----
total initialization time 1035
NODE500 2-Aug-2019 08:28:01.260 on Reigi {2}
nb>[/SHOWTOGROUPS]
Последнее редактирование:
