BobSleigh - Personal Experiences of Test Driven Development

In a previous post I commented about creating a new sourceforge project called bobsleigh (http://bobsleigh.sourceforge.net) which will be able to take a database connection, parse its schema and generate code etc from it.  I have been coding away fairly solidly on BobSleigh for a few weeks now and am close to doing a release (the current source code can be accessed using SVN).  At the start of the project I had various high level goals defined such as:  it must be simple, powerful, familiar and also completely TDD'd (test driven developed).  In this post I will outline the experiences that I encountered with creating a project that is totally TDD and some of the pitfalls / advantages I encountered.

Easily the hardest thing I found about TTD was getting started with the project.  For example, I created an empty BobSleighTest unit test class and found myself starring blankly at the screen for a few minutes...  I guess the main reason for this was at the start of the project all I had were some vague high level requirements, which in turn had to be broken down into items at a granular enough level that I could to write a test for.  Usually at the start of any project I am able to churn out code as soon as an idea comes into my head.  However, because I had to create a test first than that really forced me to think out my design.  Like most pieces of software, bobsleigh has 3 main stages: -

1. Input - BobSleigh expects input to be a "bobsleigh.xml" in the folder where it gets executed.  This is similar to ANT looking for a "build.xml" (although other filenames are allowed - this requires command line arguement's which must point to the different input filename).
2. Process - BobSleigh then parses the input file, connects and parses a database, performs validation, creates an in-memory model and parses one or more sections (similar in concept to ANT's sections).
3. Output - BobSleigh finally takes the in-memory model and delegates it to a 3rd party template engine provider (e.g. FreeMarker or Velocity) which creates source code, other files depending on what the user has defined in the various templates.

Again, this is a very high level overview of what the BobSleigh project should do.  The next question then was, which section should I concentrate on first i.e. the input, process or the output?  The option I picked was option one, 1) input, as this was the easiest one to get me started.  The bobsleigh.xml file gets parsed by BobSleighConfig.java and tested using BobSleighConfigTest.java (in the same package but in a different source folder).  It didn't take long for the code-base to grow with a test method always created before the implementation method.  To ensure code coverage, Cobertura was used.

If I could go back to the start of the project, would I use TTD again?  The answer is yes, absolutely, 100%!!!  I can't emphasise this enough and plan to use TDD as much as I can in my day job.  Before in other projects, I would use TTD now and again if it was a complex piece of code.  I would also sometimes write the unit test after the code was implemented.  This is the first project I have used TTD in all the code and the main advantages that I see are as follows: -

1. Refactoring - in my opinion, this is one of the greatest benefits of TDD if you have high test coverage (bobsleigh currently has 100% code and branch coverage in all non GUI packages).  For example, I decided to implement the ability to have multiple schema objects (instead of just one) and although conceptually it looked like a simple change, it affected every part of the system.  At each refactor I was able to run the entire suite of tests again and see exactly where things were breaking and fix each broken test one-by-one.  Once all the tests were working confidence in the system was high.
2. Better Design - again, another of the top benefits of TDD as that it forces you to think about design, even if the project is being developed in an incremental and iterative manner.  TDD seems to create classes which are simpler and less coupled to each other.
3. Unit tests help documents the code - if you are looking at a class / method in the system it is easy to see the purpose of a class / method by loading up its equivalent unit test class.  
4. Faster debugging - Another nice side effect of have good test coverage is that it is very easy to debug all methods of all your classes.  No more wasting time writing main () methods simply to create an entry point for debugging!
5. Proof that your code works - as the test suites grows (and as long as all your tests pass) you can say with confidence that your code works for what is was designed to do.

There are many other advantages to TDD and unit testing in general, but these are the main ones that I can think of.  I put refactoring at the top of the list as refactoring tends to happen at the start of a project more than when project matures and increases in size.  This is because people tend to break parts of a program when they refactor and without unit tests these broken parts go unnoticed until QA / a customer etc finds the broken code.  This in turn leads to fear of refactoring and generally only fairly simple refactors occur (e.g. class renaming using Eclipse's built-in refactoring tools).  With good code coverage, this fear is replaced with confidence and with consistent refactoring the code quality grows and grows.

Happy coding!
Bob

BobSleigh - New SourceForge Project

I recently posted about my gripes with CRUD based generation tools which come with MyBatis / Hibernate etc.  These source code / entity generators are quite powerful but fall down somewhat with providing the ability to control precisely how the source looks.  In response to this I've decided to create my first SourceForge project (since JOGRE) called "BobSleigh" (Schema Lookup & Entity Item Generator).  It is being implemented with the following goals: -

1. Simplicity - it must be simple to use i.e. no programming required for most scenarios - uses the FreeMarker template engine to generate the code.
2. Power - must be powerful i.e. have full control over how and where the source is generated.
3. Familiarity - uses XML and "borrows" concepts from ANT such as @basedir, and
4. TDD - more of a side note but so far this project has been my first 100% test driven developed project.  Cobertura is being used as to ensure code coverage.

A release should be out fairly soon but in the mean time you can download a working prototype using SVN from SourceForge.

    https://sourceforge.net/p/bobsleigh/code-0/5/tree/trunk/BobSleigh/

The following is the README file provided with the project.

                             B O B - S L E I G H

What is it?
-----------

BobSleigh - Schema Lookup & Entity Item Generator. 

BobSliegh is a Java based utility which can examine a user specified database connection and produce a simple mapped model of the database schema. This model can then be transformed into one or more files e.g. database acccess objects,

MyBatis SQL map files, Hibernate Java source etc. The utility uses the powerful FreeMarker template engine which ensures the user has totally control over the output of the generated source files. The configuration file also borrows concepts from ANT such as specifying the project "basedir" and user defined  properties.


Why?
----

Simplicity and Power!

Why use another CRUD type generation application when utilities such as MyBatis Generator exist for MyBatis or Jboss Hibernate Tools? 

When starting with a library such as MyBatis, Hiberate for the first time users run the included generation tools they are usually dismayed with the lack of control they have over the output / formatting / etc.

These tools are also product dependent e.g. MyBatis Generator will only create MyBatis artifacts.

BobSliegh takes a different approach and makes it very simple for a user to create totally customised CRUD based entity items.  A target database is parsed into a simple in-memory model which can be transformed using the very powerful FreeMarker engine either on a per table basis or on the whole database.
 
For example, there may exist a database with a single table called PRODUCT_ITEM which contains 3 fields "PRODUCT_ID", "DESCRIPTION" and "NAME".

After a database schema parse you could have the following very simple object tree: -


    table.name = "PRODUCT_ITEM";
    table.jname = "ProductItem";

    tables.columns ("PRODUCT_ID", "DESCRIPTION", "NAME")
   
    e.g. a column object will contain
        column.name = "PRODUCT_ID";     
        column.jname = "ProductId";
        column.type = "INTEGER"
        column.jtype = "int";
        column.nullable = false;
        ... etc

    tables.pkeys (list of primary key column objects)
   

This model can be used in a FreeMarker style template as follows: -


   


This will create the following POJO [ ProductItem.java ]


    package org.project.test;

    public class ProductItem {

        private int productId;
        private String description;
        private String name;

        public ProductItem () {}

        public void setProductId (int productId) {
            this.productId = productId;
        }   
        public int getProductId () {
            return this.productId;
        }   
        public void setDescription (String description) {
            this.description = description;
        }   
        public String getDescription () {
            return this.description;
        }   
        public void setName (String name) {
            this.name = name;
        }   
        public String getName () {
            return this.name;
        }   
    }
   
   
In the previous simple example the POJO's don't contain comments and only have a single blank constructor.  However, it is fairly easy for the user to update the template if the user wants a complex constructor, comments, etc and with all the power of FreeMarker.
       

The Latest Version
------------------

Details of the latest version can be found on the BobSleigh SourceForge Project web site .


Documentation
-------------

Documentation is available.


Licensing
---------

This software is licensed under the terms you may find in the file named "LICENSE" in this directory.


Enjoy !!!
Author:   Bob Marks
HomePage: http://bobsleigh.sourceforge.net

Nice little post about getting SVN (with SSH) working with Subsclipe (Eclipse plugin) - basically requires pure java enabled. http://talmai-oliveira.blogspot.com/2011/06/getting-subversion-svnssh-to-work-with.html

Using a template engine to create DAOs, SQL maps

I am a fan of iBatis / MyBatis but no one likes creating these CRUD based artifacts by hand for each table in their database!  Tools exist which code generate the CRUD stuff for you such as iBator, which I personally don't like, as you have no control over what the actual generator content looks like.

Something which could be quite nice would be to have code which parses a database schema, creates a small in memory model and using a template engine such as FreeMarker / Velocity etc generates your application DAO source files etc.  I did a quick search on google and it seems I'm not the only one who has done this ...

http://dhartford.blogspot.co.uk/2008/07/freemarker-camelcase-to-underscore.html

This could be programmed fairly quickly from scratch, a better solution may be to extend the existing iBator code generator...

JOGRE on Google App Engine

A developer Jens Scheffler is currently having a go at refactoring JOGRE to be available on the Google App Engine platform. Check out these blog post outlining his work.

http://blog.appenginefan.com/2009/05/my-summer-project-let-games-begin.html

http://blog.appenginefan.com/2009/05/initial-refactoring.html

JOGRE Beta 0.3 Released!

http://jogre.sourceforge.net

The beta release is over a year in the making and is by far the biggest release of JOGRE. There have been massive changes and additions in almost every part of JOGRE.

Eleven new games:
* Car tricks
* Chinese Checkers
* Go
* Grand Prix Jumping
* Hex
* Ninty Nine
* Points Total
* Quetris
* Texas Hold'em
* Triangulum
* Warwick

Generic abstract board components for hexagonal & triangular boards.
Player components which provide another way to join a table.
Game generator which creates the basic files, which all games use, by simply supplying a game_key and using templates. These files include Java source, ANT scripts, batch and properties files.
Much improved graphics, including a JOGRE title images which each game must have on to the top left and a game specific image on the top right.
New glass pane which can be use to draw graphics over the top of games. It is used by the API to inform users to click "start" and also displaying game over messages etc.
Full database support for persistent data in the jogre server i.e. implementation of ServerDataDB class. Database access is achieved using the iBatis library. Also if a JOGRE server is running its data persistence as a database for the first time, it will run a load script which creates tables, populates test data etc. MySQL and HSQLDB database are currently supported.
Much more unit testing i.e. 37 JUnit class containing a total of 75 classes. All database SQL queries (in iBatis maps) are fully tested in JUnit tests using the DBUnit library and an in-memory HSQLDB database. The database tests can ensure that all database SQL class are fully supported / working correctly in JOGRE. To test a new database works with JOGRE is simply a matter of updating the test database connection and returning the database test suite.
First pass at a web application, programmed using the Struts library. This is database driven (using iBatis) and allows a user to register, log on and play the games as applets.
Server administrator which replaces the old Server GUI wrapper. The administrator acts as a client, i.e. administrator must log onto a running Jogre Server with an admin username and password. The Jogre Administrator can also remotely view the current state of the JOGRE server i.e. games running, users etc. It can also remotely update the server configuration (server.xml file) in a graphical manner and view persistent data (XML and database).
Language utility to edit all the language Strings in one place (still in progress).
Updated connect 4 tutorial which shows the power of Beta 0.3.
Minor changes / improvements all over JOGRE, including the API, Server, Games etc.

Generic Online Reporting

I recently dug out my old final year project and uploaded it to my google home page to give it a permanent home.

http://sites.google.com/site/bobmarksweb/gor

The system is a web based reporting application programming entirely using Java Servlets. Database connections, queries and reports can be created using a browser using HTML forms / JavaScript. HTML based reports can then be run in a variety of languages that the user specifies.