2012-07-04

Modern C++ API in Muduo, part 1

When writing a C++ library, we have to design the interface (API) carefully [Effective C++ 3rd ed. Item 18: Make interfaces easy to use correctly and hard to use incorrectly]. Muduo uses a modern C++ API style, which consists of two aspects: callbacks and object lifetime management. This post talks about callbacks.


In traditional object-oriented C++ programs, callbacks are usually done using virtual functions. A network library would define one or more base classes, to be inherited by the user code. For example, EventHandler class or Handler class, which define some (pure-) virtual functions for event callbacks, e.g. onConnect(), onDisconnect(), onMessage(), onTimer(), and so on. In order to get event notification, user code has to inherit the base Handler, and overrides virtual functions that it cares in derived MyHandler class. Dynamic binding can only be done with pointers or references in C++, therefore the library must hold pointers or references of base Handler class, which actually refer to derived MyHandler class objects. This is legitimate in so-called pure OO languages like Java. [Java 8 will introduce closure, C# has delegate since day one.]


However in non-GC language like C++, this callback-via-virtual-function API style has a fundamental difficulty: how to manage the lifetime of MyHandler objects.
  • Is MyHandler one-per-process, or one-per-socket? Does the handler object own the socket file descriptor? If so, Handler should be non-copyable, otherwise there is a risk of double freeing the socket fd.
  • Is MyHandler object created by the library? If so, does user code need to provide a derived class of HandlerFactory and register it to the library?
  • Nevertheless, the ownership of MyHandler is vague, does the library own it?  Can library safely delete the Handler pointer it holds? Presumably the base Handler class has a virtual destructor, of course.
  • How does the library ensure that deleting the object would not cause any dangling pointers in other places of the program?
There are many traps and pitfalls like this if the C++ library interface is designed this way, see section 5 of "The Design and Use of the ACE Reactor" http://www.cs.wustl.edu/~schmidt/PDF/reactor-rules.pdf


Anyway, I’d consider ‘delete this’ a bad smell in C++ code.


In modern C++, [I meant since TR1 finalized in 2005, not C++11], there is alternative way for callbacks: std::function, aka. std::tr1::function, boost::function. Here we will use boost::function, as Muduo is written in C++03, not C++11.


Muduo library exposes all its callback functionalities using boost::function. It expects user code passing in a few boost::function s for various events, such as connection event, data arrival event, and timer event. The library keeps a copy of the boost::function passed in. When an event happens, Muduo invokes the boost::function, which in turn calls user code. Unlike MyHandler, boost::function is a value object, just like std::string and std::vector, the library can hold a copy of boost::function, no need to worry about its lifetime. A boost::function never invalidates*.


Advantages:

  • User code doesn’t need to inherit base Handler class that defined in library. There is no such base class in Muduo. Avoid the vague ownership of MyHandler objects.
  • User code even doesn’t need to use class at all. For simple network programs, plain functions are good for callbacks and event handlers (handler in general sense).
  • User code may use member functions as callbacks, using boost::bind to convert a member function and its owning object into a boost::function. The member function that receives callbacks doesn’t have to be in any particular name, onConnection, onMessage are common names though. It doesn’t need to be the same name as base class virtual function, there is no base class anyway. Moreover, its signature can be slightly varied from the boost::function declaration, taking more or less parameters.
  • Such a bound boost::function should be safe to call by library, so be careful about the lifetime of the object that the member function belongs to, try not to boost::bind a member function to plain object pointer (this pointer). Either boost::bind member function to shared_ptr of that class (or weak_ptr with a forward function), or unregister before the object being destroyed,  or boost::bind member function to long lived object that survives until process terminates.
  • Flexible. a class may let its member function A receives data from connection A, and its member function B receives data from connection B. In traditional OO design, since a class can only inherit one base class once, you will have to play some tricks to make this happen. see muduo/examples/socks4a/Tunnel.h for an example.

In general, library interfaces with boost::function put much less limitation on user code.


Worry about performance ?



Calling a boost::function could be a bit slower than calling a virtual function (at least call a pointer-to-member-function can’t be faster than call a virtual function), does it matter? I don’t think so, because IO events involve system calls. Every time the library has something to notify user code, it must have done some syscall(s), eg. accept(2) for connection callback, read(2) for data callback and connection callback, etc. A syscall traps into kernel and does context switch, which is much higher costly than a C++ virtual function call or boost::function call, anyway. So the performance overhead of boost::function is negligible.


By using boost::function as event callback mechanism of network library, the interface is much cleaner than traditional OO design IMO, and hard to use incorrectly.

This post is long enough in itself, let me talk about object lifetime management in multithreaded programs next time, thread-safe destruction is an interesting problem.


1 comment:

  1. I love your blog and style and your depth of understanding network programming ,

    I have a question, Is Muduo is thr right library which can be used for websocket server application, Also Why do you say that Muduo shouldnt be used to write http server?

    ReplyDelete