On Puzzles and Mysteries

While reading Steve Denning's  "The Leader's Guide to Radical Management" this morning, I was introduced to an interesting distinction between puzzles and mysteries. I'm not sure this would be a dictionary type of distinction but certainly a useful one. Steve asserts that a puzzle is a problem for which there is a known (or at least knowable) solution. Even though that solution may be challenging to apply it is at least plannable. Traditional management and waterfall style methods are a good fit for puzzles, not surprising as that is exactly the kind of problem that they evolved to solve.

Mysteries, on the other hand, are problems that are new. That no one has solved before. They are not really knowable (at least in advance) as they are unique. The solution is going to be hard to find and will have to be uncovered a piece at a time with each successive discovery leading the investigation like clues in a good mystery novel :)

This idea seems to be basically the same as the difference between complicated and complex in complexity theory (as I understand it). Puzzles are complicated, mysteries are complex.

In his book, Steve is asserting that the traditional methods of management are aimed at solving the puzzle of efficiently providing goods and services. We have to accept that this is a valuable puzzle to solve, it provides obvious competitive advantages. He further asserts that this does not address the issue of what goods and services to provide and that there is a gap between what customers get and what customers want that is not answered through traditional management. This is where there is an opportunity to exploit another competitive advantage by refocusing the organisation on delighting customers. This seems to be the main driving force for the radical management techniques and I have to agree that this is the new frontier where real gains in competitive advantage can be found.

Steve's book provides a lot of insight into how an organisation can refocus on this frontier; self organizing teams, leaving decisions to the last responsible moment, iterations and increments, focusing on a minimum marketable product, etc. These are the same principles (and maybe some additional ones) that come from the Agile movement and Scrum.

What interested me this morning though was the nature of this problem of delighting customers. It may be mysterious now but if the solution is Radical Management/Agile/Scrum what happens when everyone is doing it. Will the problem become a puzzle, what will the next frontier be?

I think the answer is no. Delighting customers is not the problem that is solved by these new practices, it is the result of solving some other problems. It is a moving target, the key is in finding the problems/mysteries that then lead to customer delight. Delight comes from having a problem solved that you didn't even know you had. This may be leading to a delighting customers arms race as mysteries are found and converted to puzzles but that doesn't sound so bad.

In statistics there is a concept of a non-stationary time series (I know a little about this as it was the subject of my final year dissertation at college). This is a series of data over time that is by it's nature not possible to model as it changes behaviour (technically variance) over time. Such activity is found in stock market prices where feedback in the system is a factor. While there are people making a lot of money out of quantitative analysis (predicting the future by analysing the past data) the problem is that the activity that results from applying a model may change the model.

My feeling is that delighting customers follows the same pattern. Once customers figure out what it is that's delighting them they will no longer be delighted and will look for something more/different (I hope mainly different rather than more as that's what makes chasing these problems interesting).

"Any sufficiently advanced technology is indistinguishable from magic."

Arthur C. Clarke, "Profiles of The Future", 1961 (Clarke's third law)

The problems we solve that delight people are the magic that then quickly depreciate to the status of advanced technology (sorry, advanced technology, you're just not as cool as magic). Magic (and delight) is a moving boundary.

So why am I worried about this? Well, I'm sure that there will be something new after Radical Management/Agile/Scrum but for now I wanted to reassure myself that these are the tools we need now to keep chasing the magic boundary which is still far ahead of us. When we catch it (or get close to it as traditional management has gotten us close to the magic of efficiency) we will likely need to adjust  our focus on a new magic boundary and with that will come new techniques and practices.

...

Apologies to Steve Denning if I have failed to get what he wanted to say in his book or in fact completely misrepresented him in this post. Suffice to say that everything written here is my own understanding and I do not wish to put words in anyone's mouths. Also, I'm less than a third of the way through the book so there is likely more in it than I have gotten out of it so far.

Gotcha!

So I almost ditched the OpenTV IDE again! Nothing is ever easy. I have been busy building the eclipse plugin to complement the OOOCode libraries I have been making, and everything was going great, until I tried to actually deploy the plugin in the OpenTV IDE...

Nothing, nada, zip. It just doesn't show up and nowhere does it report any errors!

All morning I spent cursing this hacked together, Frankenstein, eclipse monstrosity. This afternoon, though, I have to apologise so we can move on together in harmony.

So what's the story...

Well i'm still a bit peaved that when eclipse plugins don't load, there is no obvious error reported anywhere, but eventually I found a comment in a forum suggesting the use of the OSGI console to load the plugin and diagnose it manually. Easy, I just have to launch the OpenTV IDE with the -console option.

Sure enough a console launches alongside the editor. Next I tried...

• ? - Yay! Lot's of handy help.
• diag com.pghalliday.ooocode - Boo! bundle cannot be found
• install com.pghalliday.ooocode - Boo! invlid URL and stuff
• install file:plugins/com.pghalliday.ooocode_1.0.0.201206281825.jar - Yay! bundle installed
• diag com.pghalliday.ooocode - Oh... turns out my plugin is dependent on the org.junit bundle

So that was it. In my eagerness to have everything unit tested and to adopt test driven development I had fallen into a trap. My deployable jar file also includes the unit tests and is thus dependent on the JUnit bundle. Now I have to get Eclipse to export a jar file without the unit tests (for now i have worked around the problem by copying the org.junit bundle from the JEE Eclipse distribution to the OpenTV IDE plugins folder as well).

OOOCode - Part 2

Phew, a lot has happened since my last post on this subject. After much wrangling with the C preprocessor I have a working pattern which I'm mostly happy with. It currently includes support for the following...

• Classes
• Private data
• Public methods
• Private methods
• Multiple interfaces
• A unit test framework
• Some eclipse file templates

Still to do (in no particular order)...

• Documentation
• Performance profiling
• Exception handling
• Improve unit test automation
• Eclipse wizards

Things that I'm not quite sure about...

• Not sure if I really needed to implement the unit test stuff as classes and interfaces but it has an elegance
• Don't much like having multiple calling conventions for public methods, private methods and interface methods
• No support for inheritance - is it really needed anyway?
• No support for up casting and figuring out what type something is at run time - this may become more of an issue when thinking about exception handling
• Currently only support a single constructor

This state can be found on GitHub here

So how does it look right now (after all there is no documentation ;))

Create an application that runs unit tests

After generating your own application as described in OOOCode - Part 1 copy in the OOOCode/src/OOOCode directory from the above github project and add it and its subdirectories to the OpenTV options include paths.

Then create a Main.c as below...

	#include "opentv.h"
	#include "assert.h"
	#include "OOOUnitTestsRun.h"
	void main(void)
	{
	size_t uMemory = O_heap_available();
	OOODebug * pDebug = OOOConstruct(OOODebug);
	OOODebugReporter * pReporter = OOOConstruct(OOODebugReporter, OOOCast(OOOIDebug, pDebug));
	OOOUnitTestsRun(OOOCast(OOOIReporter, pReporter));
	OOODestroy(pReporter);
	OOODestroy(pDebug);
	assert(O_heap_available() == uMemory);
	/* Stick around so the VSTB does not exit and we know we ran everything */
	while (TRUE)
	{
	o_message tMessage;
	O_ui_get_event_wait(&tMessage);
	if (O_msg_class(&tMessage) == MSG_CLASS_CONTROL)
	{
	if (O_msg_type(&tMessage) == MSG_TYPE_QUIT)
	{
	O_exit();
	}
	}
	}
	}

view raw Main.c hosted with ❤ by GitHub

The above code achieves the following...

• Records the memory available at start up
• Creates a debug output object using the OOOConstruct macro
  • In this case an OOODebug class is used instead of direct calls to O_debug as in tests for the unit test classes themselves it is necessary to use mock objects
• Creates a debug reporter object
• Unit test reports need to go somewhere, this class dumps them to debug output
• The debug instance is cast to an IDebug interface using the OOOCast macro, in this way it is possible to pass in a mock object when needed
• Calls the OOOUnitTestsRun method passing in the reporter to run the tests
• The debug reporter is cast to an IReporter interface using the OOOCast macro, this will allow the unit test framework to be extended later with different reporter objects (eg. over HTTP, etc)
• Then the instantiated objects are destroyed using the OOODestroy macro and the memory is checked to ensure that the tests and the unit test framework used did not leak
• The last part (while loop) just ensures that the VSTB does not exit so we can see the test report in the debug output

So there are some key concepts introduced here...

• OOOConstruct - use this to construct an instance of a class, the first parameter is the class name, additional parameters are passed into the constructor as arguments
• OOOCast - use this to cast an instance of a class to an interface, the first parameter is the interface name, the second parameter is the class instance
• OOODestroy - use this to destroy an instance of a class and free it's memory, the only argument is the class instance

However, this will not yet compile. The function OOOUnitTestsRun is a special function that is generated by the OOOUnitTestsRun.h header file using xmacros. It generates an array of tests to run and runs them based on the contents of another header file: OOOTests.h

/* include test headers here */

view raw OOOTests.h hosted with ❤ by GitHub

This initial OOOTests.h is empty and so this application does not yet run any tests. Now the application can be compiled (assuming that the OOOCode source has been added to the include paths).

NB. The OOOTests.h file does not have an include guard and this is deliberate. An include guard would prevent the xmacros that use it from working. For more details on the xmacro pattern see this drdobbs article

Adding a test for MyClass

First update OOOTests.h...

#include "MyClass.Test.h"

view raw OOOTestsWithMyClass.h hosted with ❤ by GitHub

Once again this will not compile, but hey, we're doing test driven development.

Add the MyClass.Test.h header...

OOOTest(MyClass)

view raw MyClass.Test.h hosted with ❤ by GitHub

Still this will not compile, but note that the OOOTest macro call does not have a semicolon on the end - this is important. This just declares the test it does not yet compile because the test has not been implemented. Other tests can be declared with other names by adding further OOOTest calls (without semicolons)

Add the MyClass.Test.c file...

	#include "OOOUnitTestDefines.h"
	OOOTest(MyClass)
	{
	OOOInfo("MyClass test");
	}

view raw MyClass.Test.c hosted with ❤ by GitHub

Now it gets a little bit more interesting and in fact should compile and run again. When it is run this code will print the "MyClass test" string to the debug in an xml test report format signifying that it is just information. The important concepts that we now have are...

• Declare tests with calls to OOOTest in unguarded headers that are included in OOOTests.h
• Implement tests using the OOOTest macro as defined in OOOUnitTestDefines.h so that they look like functions
• We can ouput information to the test report using the OOOInfo macro (this is actually a variadic macro that behaves like printf). Two other similar macros are also available in test implementations...
  • OOOWarning - adds a warning to the test report
  • OOOError - adds an error to the test report

Adding MyClass

We are going to add a class that takes an integer in the constructor and exports a method to retrieve that integer. So lets first write some more of the test. We update MyClass.Test.c as follows...

	#include "OOOUnitTestDefines.h"
	#include "MyClass.h"
	OOOTest(MyClass)
	{
	MyClass * pMyClass = OOOConstruct(MyClass, 5);
	/* Check stuff here */
	OOOCheck(OOOCall(pMyClass, getMyField) == 5);
	OOODestroy(pMyClass);
	}

view raw MyClass.TestWithMethod.c hosted with ❤ by GitHub

Again this will not compile but we can see how we want our class to behave...

• We include the class header (does not yet exist)
• We construct an instance of the class
• We check the retrieval of the integer constructor parameter
• We destroy the instance of the class

The key concepts are...

• The memory allocated in a test must be freed in the test, the unit test framework does check for memory anomalies and adds them to the test report
• Public methods are called with the OOOCall macro, the first argument is the instance, the second argument is the method name, additional arguments would be the parameters for the method
• OOOCheck is used to test a condition that must be true for the test to pass, it can be called as many times as you like but if the condition resolves to FALSE then an error entry will be added to the test report along with the file, line and condition that failed, etc.

Add MyClass.h...

	#ifndef MYCLASS_H_
	#define MYCLASS_H_
	#include "OOOCode.h"
	#define OOOClass MyClass
	OOODeclare(int nMyField)
	OOOImplements
	OOOImplementsEnd
	OOOExports
	OOOExport(int, getMyField);
	OOOExportsEnd
	OOODeclareEnd
	#undef OOOClass
	#endif

view raw MyClass.h hosted with ❤ by GitHub

Now things are getting really interesting. Still this will not compile as we do not have an implementation for MyClass but lets go through what's happening here in the header...

• We have an include guard - that's fine here :)
• The OOCode.h header is included to enable all the OOOCode goodness ;)
• The name of the class is #defined as OOOClass - this is used inside other macros as the class name and simplifies those macro calls
• The class is declared using the OOODeclare macro, it takes the constructor arguments as parameters
• A list of implemented interfaces is given using the OOOImplements block, this must be present even if it is empty as in this case
• A list of public methods is given using the OOOExports block, in this case one method is exported using OOOExport
• The declare block is closed and importantly OOOClass is #undef'd so that other classes can be declared

Let's go through those macro calls in a bit more detail...

• OOODeclare - actually declares the type and the constructor hence the addition of the constructor arguments
• OOOImplements - starts the structure defining the public interfaces avialable (OOOImplement will be detailed later)
• OOOImplement - finalizes the interfaces structure
• OOOExports - starts the vtable structure providing access to public methods
• OOOExport - adds a public method to the vtable, the first argument is the return type, the second argument is the name of the method, any further arguments will be the parameters for the method (in this case there are none)
• OOOExportsEnd - finalizes the vtable structure
• OOODeclareEnd - this finalizes everything and defines a public structure used to access the public methods and interfaces

Now we're also ready to add the implementation, so create the following MyClass.c file...

	#include "MyClass.h"
	#define OOOClass MyClass
	OOOPrivateData
	int nMyField;
	OOOPrivateDataEnd
	OOODestructor
	{
	}
	OOODestructorEnd
	OOOMethod(int, getMyField)
	{
	return OOOF(nMyField);
	}
	OOOMethodEnd
	OOOConstructor(int nMyField)
	{
	OOOMapMethods
	OOOMethodMapping(getMyField)
	OOOMapMethodsEnd
	OOOF(nMyField) = nMyField;
	}
	OOOConstructorEnd
	#undef OOOClass

view raw MyClass.c hosted with ❤ by GitHub

So what's this all about then...

• Include the MyClass.h header
• #define OOOClass to the name of the class, again this is so that other macro calls can use it and their interfaces are thus simplified (it would be nice if macros could define other macros internally, but hey ho...)
• Declare the private data fields using OOOPrivateData - just one integer field
• Implement the destructor function with OOODestructor - in this case it is empty as no additional memory is allocated when constructing objects of this class
• Implement a method using OOOMethod - this one just returns the integer field accessed through a call to the OOOF macro
• Implement the constructor using OOOConstructor - in the constructor it is also necessary to map any internal functions to external methods and interfaces, in this case...
• OOOMapMethods is used to open a list methods to map to the exported vtable
• OOOMethodMapping is used to map the getMyField method to the first entry in the vtable - the compiler will pick up any type incompatibilities here
• The mapping is closed with OOOMapMethodsEnd
• Lastly the constructor assigns the nMyField parameter to the nMyField private data entry (again using the OOOF accessor macro)

Again let's look at these new macros...

• OOOPrivateData - starts a new private data structure, this should only appear once
• OOOPrivateDataEnd - closes the private data structure, fields in the structure should be placed between these 2 macros in the same format you would use for a struct (it is a struct!)
• OOODestructor - this starts the destructor method, destructors take no additional arguments, this should only appear once
• OOODestructorEnd - this ends the destructor method, it actually also frees the class instance which is why you don't have to do it yourself. The curly braces between these 2 macro calls in this case a re purely a matter of style and optional, they would only be required if it were necessary to declare any local variables in the destructor method. I use them anyway because it makes the implementation look more like a standard C method (a bit)
• OOOMethod - this starts a method implementation, the first argument is the return type, the second is the method name, any additional arguments will be passed into the method. The method is effectively private until it is mapped to the class or an interface vtable (the macro declares it static)
• OOOMethodEnd - this closes the method implementation, again the curly braces are mostly optional
• OOOConstructor - this starts the constructor implementation, it should appear only once. The arguments are the constructor parameters
• OOOMapMethods - this starts the class vtable mapping
• OOOMethodMapping - this maps a method to an entry in the class vtable, the only parameter is the method name - this is the private method name as defined in the call to OOOMethod, it does not have to match the exported method name in the vtable defined in the header. It is important to add the methods to the mapping in the same order that they are added to the vtable in the header using calls to OOOExport
• OOOMapMethodsEnd - this closes the vtable mapping
• OOOConstructorEnd - this closes the constructor implementation
• OOOF - this macro accesses the private fields of the current class instance, the only argument is the name of the field. It is not possible to access fields of instances of other classes but later we will see how to access fields of other instances of the same class

Adding IMyInterface

We will now add an interface that defines a single method that returns an integer. Again let's start with the test and update MyClass.Test.c as follows...

	#include "OOOUnitTestDefines.h"
	#include "MyClass.h"
	OOOTest(MyClass)
	{
	MyClass * pMyClass = OOOConstruct(MyClass, 5);
	/* Check stuff here */
	OOOCheck(OOOCall(pMyClass, getMyField) == 5);
	OOOCheck(OOOICall(OOOCast(IMyInterface, pMyClass), getData) == 5);
	OOODestroy(pMyClass);
	}

view raw MyClass.TestWithInterface.c hosted with ❤ by GitHub

Only a small change but again this will not compile...

• We added a new check, casting the instance to IMyInterface and calling the getData interface method using the OOOICall calling convention - in the test I have assumed that getData will be mapped to retrieving the constructor parameter

Only one new macro has been introduced here...

• OOOICall - this macro must be used when calling methods on interface instances, it is much the same as OOOCall in that the first argument is the interface instance, the second argument is the interface method name and any additional arguments are passed through as parameters to the implementation

To make this compile we will have to add IMyInterface.h and update MyClass to implement the interface...

	#ifndef IMYINTERFACE_H_
	#define IMYINTERFACE_H_
	#include "OOOCode.h"
	#define OOOInterface IMyInterface
	OOOVirtuals
	OOOVirtual(int, getData);
	OOOVirtualsEnd
	#undef OOOInterface
	#endif

view raw IMyInterface.h hosted with ❤ by GitHub

This is pretty similar to the pattern used to declare the class...

• There is an include guard
• OOOCode.h is included
• This time we specify the interface name in a #define called OOOInterface
• The interface vtable is then started with a call to the OOOVirtuals macro
• Methods are added to the vtable using calls to the OOOVirtual macro
• The vtable is then closed and OOOInterface is #undef'd so that other interfaces can be declared

So the new macros are...

• OOOVirtuals - starts the interface vtable
• OOOVirtual - declares a method entry in the vtable, the first argument will be the return type, the second argument is the method name and any additional arguments will be the parameters for the method. Any method implementing this virtual method will have to have the same signature and the compiler will check
• OOOVirtualsEnd - closes the interface vtable

This still won't compile so next we update MyClass.h...

	#ifndef MYCLASS_H_
	#define MYCLASS_H_
	#include "OOOCode.h"
	#include "IMyInterface.h"
	#define OOOClass MyClass
	OOODeclare(int nMyField)
	OOOImplements
	OOOImplement(IMyInterface);
	OOOImplementsEnd
	OOOExports
	OOOExport(int, getMyField);
	OOOExportsEnd
	OOODeclareEnd
	#undef OOOClass
	#endif

view raw MyClass.h hosted with ❤ by GitHub

Again a small change...

• The interface header has been added
• An OOOImplement call has been added to the OOOImplements block to add the interface to the declaration

Just one new macro then...

• OOOImplement - this adds the interface to the interface table, the only argument is the name of the interface

This will compile but the test will fail. In fact the test should crash with a NULL pointer exception as the method has neither been implemented or mapped to the interface. We need to update MyClass.c too...

	#include "MyClass.h"
	#define OOOClass MyClass
	OOOPrivateData
	int nMyField;
	OOOPrivateDataEnd
	OOODestructor
	{
	}
	OOODestructorEnd
	OOOMethod(int, getMyField)
	{
	return OOOF(nMyField);
	}
	OOOMethodEnd
	OOOMethod(int, getData)
	{
	return OOOC(getMyField);
	}
	OOOMethodEnd
	OOOConstructor(int nMyField)
	{
	#define OOOInterface IMyInterface
	OOOMapVirtuals
	OOOVirtualMapping(getData)
	OOOMapVirtualsEnd
	#undef OOOInterface
	OOOMapMethods
	OOOMethodMapping(getMyField)
	OOOMapMethodsEnd
	OOOF(nMyField) = nMyField;
	}
	OOOConstructorEnd
	#undef OOOClass

view raw MyClass.c hosted with ❤ by GitHub

Now the code will both compile and the tests will run successfully! (assuming I transcribed everything correctly).

There are 2 additions here...

• The method has been implemented as getData and this method just calls the other method using the private method calling convention, OOOC
• The interface vtable has been mapped in the constructor
  • The interface name is given in the OOOInterface #define to simplify the other macro calls
  • The interface vtable mapping is started with a call to OOOMapVirtuals
  • The getData method is mapped using a call to the OOOVirtualMapping macro
  • The mapping is closed with a call to OOOMapVirtualsEnd and OOOInterface is undef'd so that other interfaces can be mapped

So the new macros we have used are...

• OOOC - this macro accesses the private methods of the current class instance, the first argument is the name of the method and any additional arguments are passed through as parameters to the method. It is not possible to access private methods of instances of other classes but later we will see how to access methods of other instances of the same class
• OOOMapVirtuals - this starts the interface vtable mapping
• OOOVirtualMapping - this maps a method to an entry in the interface vtable, the only parameter is the method name - this is the private method name as defined in the call to OOOMethod, it does not have to match the exported method name in the vtable defined in the interface header. It is important to add the methods to the mapping in the same order that they are added to the vtable in the interface header using calls to OOOVirtual
• OOOMapVirtualsEnd - this closes the interface vtable mapping

Adding copy and isEqual methods

As a final example and to round out the macro examples let's see how we can add additional methods to copy and compare instances of MyClass. Of course, we start with a test so let's update MyClass.Test.c...

	#include "OOOUnitTestDefines.h"
	#include "MyClass.h"
	OOOTest(MyClass)
	{
	MyClass * pMyClass = OOOConstruct(MyClass, 5);
	MyClass * pMyClassCopy = OOOCall(pMyClass, copy);
	/* Check stuff here */
	OOOCheck(OOOCall(pMyClass, getMyField) == 5);
	OOOCheck(OOOICall(OOOCast(IMyInterface, pMyClass), getData) == 5);
	OOOCheck(OOOCall(pMyClassCopy, isEqual, pMyClass));
	OOOCheck(OOOCall(pMyClassCopy , getMyField) == 5);
	OOOCheck(pMyClassCopy != pMyClass);
	OOODestroy(pMyClass);
	OOODestroy(pMyClassCopy);
	}

view raw MyClass.Test.c hosted with ❤ by GitHub

The following changes were made...

• A new instance of MyClass, pMyClassCopy, is generated through a call to a new copy method
• We check that the new copy is equal to the original
• We check that the new copy returns the same value from getMyField
• We check that the copy method didn't cheat and that the copy is a different instance (pointer address)
• We remember to clean up the new instance too

Once again our code does not compile, but that's ok. We need to update MyClass.h to export the new methods...

	#ifndef MYCLASS_H_
	#define MYCLASS_H_
	#include "OOOCode.h"
	#include "IMyInterface.h"
	#define OOOClass MyClass
	OOODeclare(int nMyField)
	OOOImplements
	OOOImplement(IMyInterface);
	OOOImplementsEnd
	OOOExports
	OOOExport(int, getMyField);
	OOOExport(MyClass *, copy);
	OOOExport(bool, isEqual, MyClass * pCompare);
	OOOExportsEnd
	OOODeclareEnd
	#undef OOOClass
	#endif

view raw MyClassWithCopy.h hosted with ❤ by GitHub

This will now compile but, as with the interface implementation, the test will crash when it gets to the copy call as the method has not been implemented and mapped in the vtable. Anyway let's see what we've done...

• Two new calls have been made to OOOExport to export the copy and isEqual methods

Now we implement and map the methods in MyClass.c...

	#include "MyClass.h"
	#define OOOClass MyClass
	OOOPrivateData
	int nMyField;
	OOOPrivateDataEnd
	OOODestructor
	{
	}
	OOODestructorEnd
	OOOMethod(int, getMyField)
	{
	return OOOF(nMyField);
	}
	OOOMethodEnd
	OOOMethod(MyClass *, copy)
	{
	return OOOConstruct(MyClass, OOOF(nMyField));
	}
	OOOMethodEnd
	OOOMethod(bool, isEqual, MyClass * pCompare)
	{
	return (OOOF(nMyField) == OOOPCall(pCompare, getMyField));
	}
	OOOMethodEnd
	OOOMethod(int, getData)
	{
	return OOOC(getMyField);
	}
	OOOMethodEnd
	OOOConstructor(int nMyField)
	{
	#define OOOInterface IMyInterface
	OOOMapVirtuals
	OOOVirtualMapping(getData)
	OOOMapVirtualsEnd
	#undef OOOInterface
	OOOMapMethods
	OOOMethodMapping(getMyField),
	OOOMethodMapping(copy),
	OOOMethodMapping(isEqual)
	OOOMapMethodsEnd
	OOOF(nMyField) = nMyField;
	}
	OOOConstructorEnd
	#undef OOOClass

view raw MyClassWithCopy.c hosted with ❤ by GitHub

Yay, sucess! The code compiles, runs and the tests pass. We added 2 new methods and mapped them so what's new in this...

• In the compare method we used a new macro, OOOPCall, this is more efficient than OOOCall and can also be used to access unmapped methods in a class. The first argument is the class instance, the second is the method name and any additional arguments will be passed into the method. In this case OOOC could not be used as we wanted to call a method on another instance
• Notice that the additional mappings in OOOMapMethods are preceded by commas - this is because they result in static initialiser elements in a structure. The same applied to the virtual mappings if there are more than one.

So what haven't we seen? Well, two additional things spring to mind...

• It is also possible to access fields on other instances of a class. This is achieved through calls to OOOField, like OOOPCall this can only be used in the class implementation and the first parameter will be the instance, the second parameter is the field name. We could have used this in place of OOOPCall above but it is a matter of style to use the accessor method instead (performance optimisations could dictate otherwise though)
• If it is necessary to access the current instance (perhaps to return from a method or pass into another method) then it is always available in the methods, constructor and destructor through the OOOThis pointer

So that's it. Although this will all probably change in the next 5 minutes. If you're interested then keep an eye on this blog and the GitHub repository.

A Macro Interlude

Or... Traversing Macro Pasting Hell

Full disclaimer: This article used to be very different but was also complete tosh. The code I wrote based on its assumptions and misinformation was working fine for days and then broke around 5 minutes after posting the article when I discovered a new use case. This new article addresses that use case and the subsequent solution

Macro pasting

The C preprocessor has a handy (if not essential) operator for pasting 2 tokens together, ##. I use it all over the place in my OOOCode project to generate class and function names, etc using macros. It can be used like this...

	#define PASTE(ARG1, ARG2) ARG1 ## ARG2
	PASTE(Hello, World)();
	/* expands to */
	HelloWorld();

view raw Paste.c hosted with ❤ by GitHub

Now consider the following...

	#define PASTE(ARG1, ARG2) ARG1 ## ARG2
	#define HELLO Hello
	PASTE(HELLO, World)();
	/* expands to */
	HELLOWorld();

view raw PasteBroken.c hosted with ❤ by GitHub

In this second example I have passed a macro in as an argument to the PASTE macro and as a result it does not get expanded. In order to fix this it is necessary to add a level of indirection...

	#define _PASTE(ARG1, ARG2) ARG1 ## ARG2
	#define PASTE(ARG1, ARG2) PASTE(ARG1, ARG2)
	#define HELLO Hello
	PASTE(HELLO, World)();
	/* expands to */
	HelloWorld();

view raw PasteFixed.c hosted with ❤ by GitHub

As an aside, this same problem (and solution) occurs with the quoting operator, #, too.

Variadic macros and swallowing extra commas

Now it gets interesting as the pasting operator can also be used in variadic macros to swallow commas when no arguments are provided...

	#define CALLONE(FUNCTION, ARGS...) FUNCTION(1 , ##ARGS)
	CALLONE(myFunc);
	/* expands to */
	myFunc(1);

view raw VariadicMacro.c hosted with ❤ by GitHub

Handy, yeah? Well sort of. The problem is that this still exhibits the same problems as above when macros are used as arguments...

Macro pasting variadic hell

	#define CALLONE(FUNCTION, ARGS...) FUNCTION(1 , ##ARGS)
	CALLONE(add, CALLONE(addOne));
	/* expands to */
	add(1, CALLONE(addOne));

view raw FailsToExpand.c hosted with ❤ by GitHub

The above code does not compile as the second macro call does not get expanded. So I tried this...

	#define _CALLONE(FUNCTION, ARGS...) FUNCTION(1 , ##ARGS)
	#define CALLONE(FUNCTION, ARGS...) _CALLONE(FUNCTION, ARGS)
	CALLONE(add, CALLONE(addOne));
	/* expands to */
	add(1, addOne(1, ));

view raw FailsToSwallowComma.c hosted with ❤ by GitHub

I'm not sure if this would work in other environments but the C Preprocessor that comes with the OpenTV IDE doesn't swallow the comma in this case.

This gave me a big problem. I can either support macros as arguments or zero length argument lists... but not both :(

Believe me I tried a great many more constructions involving the ## operator and various indirections but to no avail. It just wasn't happening. Eventually (it was quite long time that may even have involved praying as I was a long way into my OOOCode stuff and this was pretty key) I came across a different solution involving detecting empty argument lists. Doing this is not simple and definitely not something I want to get into here, but just know that in the following example the ISEMPTY macro expands to 1 if the argument list supplied is empty or 0 if not...

	#define ISEMPTY(ARGS...) /* Too complicated to put here, see http://gustedt.wordpress.com/2010/06/08/detect-empty-macro-arguments/ */
	#define _PASTE(ARG1, ARG2) ARG1 ## ARG2
	#define PASTE(ARG1, ARG2) PASTE(ARG1, ARG2)
	#define CALLONE0(FUNCTION, ARGS...) FUNCTION(1, ARGS)
	#define CALLONE1(FUNCTION, ARGS...) FUNCTION(1)
	#define CALLONE(FUNCTION, ARGS...) PASTE(CALLONE,ISEMPTY(ARGS))(FUNCTION, ARGS)
	CALLONE(add, CALLONE(addOne));
	/* expands to */
	add(1, addOne(1));

view raw Solution.c hosted with ❤ by GitHub

For the ISEMPTY macro stuff, special thanks have to go to Jens Gustedt and this article:

http://gustedt.wordpress.com/2010/06/08/detect-empty-macro-arguments/

OOOCode - Part 1

This is a continuation of the following post on Object Oriented C

Object Oriented C - Part 2

First thing is to get the existing pattern example into an OCode project and stick that up on GitHub. I don't like the default OpenTV IDE project templates as they actually require multiple Eclipse projects and due to the nature of the OpenTV hacked Eclipse cannot be built and run from a command line. This creates obvious problems when trying to set up a continuous integration environment. However for these purposes a standard OCode template will suffice.

To do this:


I created an OOOCode directory and checked out the OOOCode git repository to it.

I opened the OpenTV IDE and selected  the new OOOCode directory as the workspace location

I created a new OpenTV Project called OOOCode in the default location (pressing Finish immediately as I did not want the hello world example code)

This resulted in the following projects being created in my OOOCode directory.

• OOOCode
• OOOCodeDir
• OOOCodeFlow
• OOOCodeRes

As pictured:

To build and run the project in the OpenTV Virtual STB. 

Choose "Build All" from the "Project" menu.

Select the OOOCodeFlow project and press the Run button.

In the next dialog choose "Local OpenTV Application".

That's as much as I'm going to say about the OpenTV IDE for now but next I copied in my existing OO pattern example code and committed it to GitHub. This starting state can be found here:

https://github.com/pghalliday/OOOCode/tree/9fd2a18d20f5b515a9951a2e3b1c15af06a0ecd6

Full disclosure... I did fix a couple of issues in the code i originally posted in the gists:

• I missed a parameter in the MyClass constructor
• I forgot that the VSTB exits when the main method exits and as such it can be difficult to check the debug output unless you put a real message loop in and wait for the quit message

Object Oriented C - Part 2

Now that I have detailed my existing pattern it's time to start working on an easier to implement/maintain replacement. To do this I need to start thinking about how to test it. After all I am fully embracing test driven development these days.

One thing I forgot of course when detailing the existing pattern is how to use it in an application. I think that part at least is pretty straight forward but just to complete the square, here's an example main.c:

	#include "opentv.h"
	#include "assert.h"
	#include "MyClass.h"
	/* This method just waits for a quit message and then exits the application */
	static void Main_waitToExit(void)
	{
	while (TRUE)
	{
	o_message tMessage;
	O_ui_get_event_wait(&tMessage);
	if (O_msg_class(&tMessage) == MSG_CLASS_CONTROL)
	{
	if (O_msg_type(&tMessage) == MSG_TYPE_QUIT)
	{
	O_exit();
	}
	}
	}
	}
	void main(void)
	{
	/*
	record the current memory available so we
	can detect memory leaks
	*/
	size_t uMemory = O_heap_available();
	MyClass * pInstance = MyClass_create(5);
	MyInterface * pInterfaceInstance = MyClass_asMyInterface(pInstance);
	/* Method tests */
	assert(MyClass_getMyField(pInstance) == 5);
	assert(MyInterface_myMethod(pInterfaceInstance, 3) == 8);
	/* Memory leak test */
	MyClass_destroy(pInstance);
	assert(O_heap_available() == uMemory);
	/* Stick around so the VSTB does not exit and we know we ran everything */
	O_debug("OOOCode: Tests passed\n");
	Main_waitToExit();
	}

view raw Main.c hosted with ❤ by GitHub

So here are the tests although I haven't used the unit test framework I might use ordinarily (I didn't want to cloud the details and introduce more dependencies). Again I haven't run this code (currently working in OSx and the OpenTV IDE only runs on Windows) but as soon as I do I'll correct any mistakes found and likely add it to my previously created project in GitHub (currently lying empty)

One last thing for this post though, I thought of a better name for my project. From Object Oriented OCode we get OOOCode. That's right... Oooooo, code ;) (or Oh! Oh! Oh! Code - haven't decided)

Previous link is now almost certainly broken so here's a new one.

https://github.com/pghalliday/OOOCode

Object Oriented C Reboot

As is normally the case I started something and then put it down and did lots of other things instead. In a rare bout of refocusing though I'm picking up the object oriented C (OCode) stuff again.

Previously I got very side tracked into setting up a perfect development environment as I would like to use at work. Something that would be compatible with automated build systems. Furthermore, I started trying to port the subversion externals pattern that I like to use for shared code to Git on GitHub - I quickly discovered that I'm still very much a Git newbie. This was not very lean of me...

So i'm rebooting. This time around I will focus on the task at hand which is to create a simple templating and/or preprocessing system for generating class and interface boiler plate for use in OpenTV applications which are exclusively written in C.

Let's start with a description of the pattern I currently use to implement classes. This part is simple and the boiler plate is not so bad.

The header:

	#ifndef MyClass_H
	#define MyClass_H
	typedef struct _MyClass MyClass;
	extern MyClass * MyClass_create(int nMyField);
	extern void MyClass_destroy(MyClass * pThis);
	extern int MyClass_getMyField(MyClass * pThis);
	#endif

view raw MyClass.h hosted with ❤ by GitHub

The implementation:

	#include "MyClass.h"
	#include "opentv.h"
	struct _MyClass
	{
	int nMyField;
	};
	MyClass * MyClass_create(int nMyField)
	{
	MyClass * pThis = O_malloc(sizeof(MyClass));
	pThis->nMyField = nMyField;
	return pThis;
	}
	void MyClass_destroy(MyClass * pThis)
	{
	O_free(pThis);
	}
	int MyClass_getMyField(MyClass * pThis)
	{
	return pThis->myField;
	}

view raw MyClass.c hosted with ❤ by GitHub

Excuse any obvious errors, I just typed that out without trying to run or compile it, but i think you can get the idea - nicely encapsulated, right?

So that's all good. A little boiler plate in exposing opaque types and constructors/destructors but not so bad. The problems/challenges(/opportunities ;)) start when I try to extend this pattern with interfaces. Let's extend the example with that pattern.

The interface:

	#ifndef MyInterface_H
	#define MyInterface_H
	#include "opentv.h"
	/*
	Prototype a method signature to be implemented for
	this interface. Allows the compiler to spot errors
	but can't get round the void pointer for the instance
	*/
	typedef int (* MyInterface_MyMethod)(void * pInstance, int nArgument);
	/* Interface structure (vtable?) */
	typedef struct
	{
	void * pInstance;
	MyInterface_MyMethod cbMyMethod;
	}
	MyInterface;
	/*
	Interface constructor. I use static inline for all
	interface methods so that they compile out much like
	preprocessor stuff but also the compiler will help me
	with type checking, etc
	*/
	static inline MyInterface * MyInterface_create(void * pInstance, MyInterface_MyMethod cbMyMethod)
	{
	MyInterface * pThis = O_malloc(sizeof(MyInterface));
	pThis->pInstance = pInstance;
	pThis->cbMyMethod = cbMyMethod;
	return pThis;
	}
	/* Interface destructor */
	static inline void MyInterface_destroy(MyInterface * pThis)
	{
	O_free(pThis);
	}
	/*
	This method allows the implementation of MyInterface_MyMethod
	to be called thus facilitating polymorphism which is our end goal
	*/
	static inline int MyInterface_myMethod(MyInterface * pThis, int nArgument)
	{
	return pThis->cbMyMethod(pThis->pInstance, nArgument);
	}
	#endif

view raw MyInterface.h hosted with ❤ by GitHub

Now we can see some really obvious complexity. Just look at the length of it and it doesn't even do anything really. Immediately apparent is how much work it would be to add a new method to the interface.

1. Add a new typedef for the method prototype
2. Add a field to the interface structure
3. Add an argument to the interface constructor
4. Add a redirector method to allow the implementation to be called

It's fiddly work and potentially error prone (and this stuff can be hard to debug). The good news is that interfaces tend to be fairly stable once done as they don't contain business logic (although they may represent it I guess).

We're not finished though. The interface has to be implemented by our class for it to be useful.

Here's the new header for our class:

	#ifndef MyClass_H
	#define MyClass_H
	/* Need to include the interface header */
	#include "MyInterface.h"
	typedef struct _MyClass MyClass;
	extern MyClass * MyClass_create(int nMyField);
	extern void MyClass_destroy(MyClass * pThis);
	extern int MyClass_getMyField(MyClass * pThis);
	/* New method to "cast" MyClass to MyInterface */
	extern MyInterface * MyClass_asMyInterface(MyClass * pThis);
	#endif

view raw MyClassWithMyInterface.h hosted with ❤ by GitHub

Notice the addition of a method to get an instance as an instance of MyInterface this is our casting convention.

Here's the new implementation of our class:

	#include "MyClass.h"
	#include "opentv.h"
	struct _MyClass
	{
	/* Need a field to store the interface instance */
	MyInterface * pMyInterface;
	int nMyField;
	};
	/*
	private implementation of a MyInterface_MyMethod method - we keep
	the void pointer parameter so that we don't have to cast the method
	when passing it into the interface constructor. Doing so could hide nasty
	type errors further down the road.
	*/
	static int MyClass_myMethod(void * pInstance, int nArgument)
	{
	/*
	this #define is an optimisation that I use instead
	of adding a local variable to the stack. It's pretty
	optional but I worry about this sort of thing when
	running code on low horsepower set top boxes
	*/
	#define pThis ((MyClass *) pInstance)
	/* ooh, look - that's what myMethod is for ;) */
	return pThis->nMyField + nArgument;
	/*
	let's not forget to undef pThis we may want to use
	that name a lot in other methods ;)
	*/
	#undef pThis
	}
	MyClass * MyClass_create(int nMyField)
	{
	MyClass * pThis = O_malloc(sizeof(MyClass));
	/* Construct an interface instance */
	pThis->pMyInterface = MyInterface_create(pThis, MyClass_myMethod);
	pThis->nMyField = nMyField;
	return pThis;
	}
	void MyClass_destroy(MyClass * pThis)
	{
	/* don't forget to destroy the interface instance */
	MyInterface_destroy(pThis->pMyInterface);
	O_free(pThis);
	}
	int MyClass_getMyField(MyClass * pThis)
	{
	return pThis->nMyField;
	}
	/* Casting convention implementation */
	MyInterface * MyClass_asMyInterface(MyClass * pThis)
	{
	return pThis->pMyInterface;
	}

view raw MyClassWithMyInterface.c hosted with ❤ by GitHub

That wasn't so bad, we:

1. Added a field to the class structure to store the interface instance
2. Implemented the interface method
3. Constructed an interface instance in the class constructor
4. Destroyed the interface instance in the class destructor
5. Added the method to implement our casting convention

Again it was fiddly though and remembering that we have to do these things in every class that implements the interface we are now exposed to following types of errors:

• Memory leaks due to forgetting to destroy the interface
• Strange affects from casting incorrectly in method implementations (doesn't seem likely until you remember that casts are a nightmare for hiding copy paste errors)

So let's review. We now have 3 quite complicated files that are quite hard to maintain. Particularly, changes in interfaces result in a large quantity of refactoring radiating out all over the place. Plus we have to be careful whenever we implement a new instance of an interface. And remember that this example only implements 1 interface method!

This is a barrier to using the pattern which I would like to overcome. Just writing it up took longer than I expected and I have to go out now so I guess there will be a part 2 where I actually get started on how I would like it to look and work :)

...
...
...

As an aside the work I have been doing to make OpenTV application generating and testing makefiles using make function implementations will likely be the subject of a future post. As will figuring out how to share code and resources across Git repositories/projects.

Required watching!

http://jamesshore.com/Blog/Lets-Play/

Well it might be if it weren't 197 episodes and counting but this series of videos illustrating test driven development (TDD) in Java and Eclipse are really excellent. I'm currently up to episode 25 (since yesterday... addicted!) and already I have learnt so much about:

• writing tests up front
• emergent design
• AND just how magical eclipse is when it comes to generating code and refactoring in Java

I really recommend watching some of these. It starts with a completely blank application so I would go from there.

I really want some of these IDE features in the OCode development we do too. And I can see that there may be some mileage in that.

Now i can't wait for James' new series that I gladly helped fund on Kickstarter.

http://www.letscodejavascript.com/

2 things that have got me excited recently in one package - TDD and server side Javascript... Yay, Node.js!

Error handling in C

An interesting and somewhat familiar set of macros for error handling in C:

http://c.learncodethehardway.org/book/learn-c-the-hard-waych21.html

A possible coding standard. Not with the debug flags (I hate debug builds!) but interesting all the same. I can see this approach getting rid of some of the over indentation caused by checking return values. As a coding standard though, I am resistant to always using the return value of a function as an error code... maybe I shouldn't be.

The most interesting part is that it looks and behaves like exception handling to an extent. A problem which is altogether too difficult to mess around with in C. However by stopping short of trying to automatically pass you back up the stack perhaps this does capture some of the essence (read value). A good, lean application of the 80/20 rule.

Still, to retrofit this stuff would likely fall short of being lean. Maybe I can mix it into the the Object Oriented C stuff I started working on.

Object Oriented C

I'm waiting for virus updates to complete and the new windows GitHub client to install - why is my computer so slow today!?

Anyway a chance to write a blog post then.

I'm becoming increasingly agile :) ... no not like a cat :s

Recently I completed my certified Scrum Master and Product Owner training and we're making great progress at work putting it all into practice. It really has reinvigorated my interest in software development and once again I am reading voraciously on subjects that actually excite me (you'll maybe shake your head when you realise what those subjects are ;)). Can't remember having this feeling since I first started coding back in University.

So what's on my mind today?

Well, I've been reading Michael Feather's "Working Effectively With Legacy Code" and once again I have found an author who really does reflect, distill and reinforce what I only really knew at the back of my mind. This is great as it gives me more confidence that I'm on the right path but also provides a glimpse of where that path leads.

One place that it leads is to greater use of object oriented principles in our code.

Some background... We write software for the OpenTV middleware in C to run on digital TV set top boxes. This is not object oriented. It isn't even really C, we can't use standard libraries and have to write pretty much everything from scratch. This has included basic libraries and a lot of tools and frameworks. Our main focuses at the moment are

• Reduce build times and complexity
• Use more off the shelf and advanced development/debugging tools
• There is a freely available OpenTV IDE based on Eclipse which is very useful as it also includes a set top box emulator allowing us to step through code (http://community.opentv.com)
• Automate builds / implement continuous integration (Jenkins)
• This conflicts with the eclipse IDE which is not good with headless builds, we've worked around this though, it just means we can't take advantage of standard OpenTV project templates. No biggy, we know what we're doing :)
• Add a unit testing framework and increase test coverage of legacy code
• Test driven development
• We have the framework so we have to do this to really reap the benefits in my opinion
• Automate tests
• Very much a TODO - the set top box emulator does not like to run on our continuous integration server unless it is not running as a service and some one is logged on. It uses DirectX :s
• Move away from specification documents to specification by tests.
• We are looking at integrating Fit and FitNesse to capture business rules and automate acceptance testing

So why more OO?

Simple really. We need units to test. We need those units to be independent so that we can develop and verify them quickly. The approaches of TDD and the means of dealing with legacy code to make it more maintainable/manageable require great discipline and effort. We need to make the task less onerous. A major part of doing that is to use interfaces to break dependencies between libraries. There are other options in C (preprocessor for example) and we have to be careful about performance on our platform. But with careful application I am convinced that we will get the best mileage out of a consistent application of OO interfaces.

So today I am going to figure out a good standard pattern for implementing classes and interfaces in C. Probably using the preprocessor to do some of the hard work. Previous attempts have always resulted in a lot of unwieldy repetition that I sometimes think only I understand (making it instant legacy code!).

Ok, installs completed :) New git repository created. I wonder if I'll just end up reinventing Objective C.