Coding Clarity

The Maritime Dev Con was a huge success. About 95 people total attended the event making it a huge success for developers in the Maritimes. I had a great time at the event and met a bunch of really cool people.

The presentations I gave went well with a number of attendees. I’m putting up the slides from the presentation here in case you want to review them.

I’m also going to include the sample code from the presentations. The samples was for the modern hello world example in Java and Groovy. It used the twitter API to query MaritimeDevCon from twitter to find my ‘Hello World’ tweet.

Modern Java Development

Slides: MaritimeDevCon2010 – Java Jumpstart

package com.chrisdail.monctondevcon;

import java.util.List;
import twitter4j.*;

public class ModernHelloWorld {
    public static void main(String[] args) throws TwitterException {
        Twitter twitter = new TwitterFactory().getInstance();
        Query query = new Query("MaritimeDevCon");
        List<Tweet> tweets = twitter.search(query).getTweets();
        for (Tweet tweet : tweets) {
            System.out.println(tweet.getFromUser() + ": " + tweet.getText());
        }
    }
}

Groovy Primer

Slides: MaritimeDevCon2010 – Groovy Primer

@Grab("org.twitter4j:twitter4j-core:2.1.0")
import twitter4j.*

def twitter = new TwitterFactory().instance
twitter.search(new Query("MaritimeDevCon")).tweets.each {
    println "$it.fromUser: $it.text"
}

There is going to be a maritime developers conference coming up on June 18th in Moncton. It is going to be a great opportunity to have developers from Moncton and other areas of the maritimes get together and learn a bit about other languages and technologies they might not have been exposed to. All of the presentations are limited to 45 min and will mostly give an introduction to the language or technology.

Information about the DevCon can be found here: http://careertown.ca/devcon/.

I will be giving two presentations at this conference.

• Modern Java Development – In this presentation I’m going to give an introduction to Java for non-Java developers. It will cover the basics of knowing where to start and getting started.
• Groovy Primer – This is essentially going to be a rehash of the Groovy talk I gave at the Maritime Java User’s Group a month or so ago. This will focus on showing what Groovy has to offer (particularly to Java developers) and how to get started with Groovy.

Hope to see you there!

Problem. Bugs happen. The common solution to this problem is to fix the bug and release a patch. Version 1.0 has bugs, version 1.0.1 fixes those bugs.

Inevitably at some point in time you will need to put together a list of all of the changes in a release. For me, this needs to go into a format we can post on our wiki. This process can be tedious if it is a manual process. There are a few approaches to handling this. You can go against the bug tracking repository and look for what bugs were fixed for this release. This will tell you everything that should have changed. I say ’should have’ here because you cannot know for sure if the information is 100% accurate.

The other option is to go to the version control repository for information on what has change. This is the authoritative source of what has changed but often contains more information than what you would want in a change-log.

In my previous post on version control I mentioned that we have best practices around format for commit messages. All bugs start with the words “Bug

I wrote a simple groovy script that generates this change-log for me. It uses the subversion command line interface. The starting point is using the 'svn log' command. This command generates a change-log. Here are the options used to this command:

• The ‘–xml’ option is used to format the output as XML. This allows groovy to break it down easily.
• The ‘-g’ option is used which shows log messages from other revisions that were merged onto this branch. Let’s say you have 50 bugs that are merged onto the bug fix branch all at once. This would create a single revision on the branch. Using this option includes all 50 comments from their original commit on the trunk. This detail we want in the change-log. This gives nested entries though so the code has to handle that case.
• The ‘-r’ option is used to specify the revision range to use. In this case for a branch, we want from the previous release revision number to the current (or HEAD). For this example, let’s assume the 1.0 branch was at revision 1528.

The command to run then becomes:

svn log -r HEAD:1528 -g --xml

The next step that needs to be done is to take this XML and turn it into a change-log. I plan to use this as a comment into a wiki so I prefix the lines with ‘*’ so they will appear as a bulleted list in trac. It also puts the revision number at the end of the line in brackets. The output should look like this:

 * Bug 123: Fixed some bug (1554)
 * Bug 126: Some other issue (1588)
 * Bug 322: Fixed the thing (1600)

To generate this changelog, I wrote a groovy script. It uses the svn command to generate the changelog and uses Groovy’s XML Parsing to break it up and format it. The path to the working directory and revision number would change from release to release but the rest of the code is reusable.

def handleEntry
handleEntry = {entry->
    def message = entry.msg.text()
    if (message.size() >= 3 && message[0..2].equalsIgnoreCase("bug")) {
        println " * $message (${entry.@revision})"
    }
    entry.logentry.each(handleEntry)
}

def proc = "svn log -r HEAD:1528 -g --xml".execute(null, new File("/path/to/working/directory"))
new XmlParser().parseText(proc.text).logentry.each(handleEntry)

Two of the most useful tools to a developer outside of their development environment are version control and bug tracking systems. Version control allows tracking of changes to the product and allows for branching and merging. Bug tracking systems allow for tracking issues with the product whether they be bugs or enhancements.

Even though these tools are often separate products, they have a major commonality which is the code you are working with. Often times you want to be able to see for any given bug number, what code was changed for that bug. Also, for a change in the code (in version control) you want to see if it was associated with a particular issue in the bug tracking software.

At the company I work for we use Subversion for version control and Bugzilla for bug tracking. We have some best practices around these tools to make things easier.

Version Control and Bug Tracking Best Practices

When resolving issues in the bug tracking database, our team always puts in the build number of the build that contains the fix. This way a person who is looking at the bug can know if the build they have contains the fix. Anytime our team fixes a bug we put in a comment that looks like this:

Build Fixed: 1.0.1.12354

The last number is the revision number in Subversion.

When we commit code changes to Subversion, we also include the bug number for the bug being fixed. Our commit messages always appear in this format:

Bug 1234: Fixed this bug

Subversion Tooling

Recently I came across a neat feature in Subversion that allows you to link it to a bug tracking system. Basically this allows clicking on the bug number in the subversion history view to take you directly to the bug number in the bug tracking software.

Enabling this feature is fairly simple to do and involves setting 2 properties in the subversion repository. These properties need to be set on the root folder in subversion that you would use to checkout your project from. It automatically is available for everything in that tree but you need to checkout from this root for it to work. These are the two properties that need to be set.

• bugtraq:logregex – This defines a regular expression to ‘match’ bug numbers in subversion comments. For the pattern I listed above, we are using: [Bb][Uu][Gg] (\d+)
• bugtraq:url – This defines a URL to go to when the user clicks on a bug number. The browser is launched when the number is clicked on and takes you to this URL replacing the BUGID parameter. For our bugzilla repository we are using: https://some.server.somewhere.localhost/show_bug.cgi?id=%BUGID%

The following steps walk through this process of how to set this up using Tortoise SVN:

• On the root folder of your subversion working copy, right click on the folder and click TortoiseSVN -> Properties.

  

• Add each property listed above as new properties to the list.

  

I have been reading a lot lately about people hating on SOAP based web services. As a whole, the Web is moving more toward REST based APIs. This post is to make a case for WSDL and SOAP based web services.

Don’t get me wrong. I’m a huge fan of RESTful Web Services. I use them in many places where they make sense in the software I develop. I am not writing this post to say that SOAP/WSDL based web services are better than the REST style but I intend to point out some of the things that a WSDL does better.

It all comes down to perspective. Who is going to be consuming the Web Service? Is the consumer going to be a human or a machine? Let’s look at each case.

• Humans Consuming Web Services. By humans consuming web services, I mean a programmer sits down and writes some code to use a web services. The common example of this is a developer from a website using the Flicker, Twitter, Google Maps or Facebook API to integrate with their site. Even in the business world, when a user needs to write a piece of code to connect two things together, REST is the clear winner. REST style web services are easier to work with and usually result in much cleaner code.
• Machines Consuming Web Services. By machines consuming web services, I’m referring to software that interprets and leverages the web services. The primary examples for this are Business Process Management and Runbook Automation Software. This happens to be the area in which I develop software. In this space the goal is to allow machines to interpret the Web Services (or other technologies) and allow the user to just map from service to service. The user needs to know nothing about the transport or how the service function themselves as the machine is responsible for all of that. This type of software is typically used in the business world and not on the Web.

  For this type of software, a strictly defined specification (a WSDL) provides a valuable tool to the software that needs to interpret it. A REST web service may be easier for a human to use by reading documentation, but we have yet to make a computer read documentation and produce results. Also, many REST style web services have XML formats that cannot be expressed in a Schema. This may be fine for a human, but XML Schemas provide an easy way for a computer to consume and understand the XML format. Yes, XML Schemas are very wordy and difficult to read by a human. They are also a pain to write properly. Even with their limitations, they do a good job of defining XML in a way that a machine can interpret them.

I see a lot of newer programmers who have only ever done work on the Web claim that REST is the best choice in all cases. We have graduated beyond SOAP to the superior technology. Even Joel Spolsky mentioned it on episode 64 of stackoverfow. REST as a superior technology is simply not true. REST may be the best choice for the Web but there are many other uses for Web Services besides the Web.

REST is getting closer to what WSDL has to offer. With WSDL version 2.0 or WADL you can define REST style web services. Maybe in a few years things will be different. Maybe we will get to the point where REST really is better than traditional web services. But we are not there yet.

The Zune Bug

A few weeks ago I heard of this issue where the Microsoft Zune crashes and won’t startup on December 31, 2008. The reason for this? A bug in the software that handled leap years. There are lots of articles on the original issue.

When I heard of this issue, I originally thought ‘How could that happen?’. Sure I could understand not handling things correctly when it is a leap year but causing a crash?

Well, if you like me were wondering how that could happen, you can now find out for yourself. This is a post someone made of the actual code that runs on the Zune. Look at line 249 and on.

If you missed it, there is a bug (obviously) where if the number of days passed in is 366, which is the case for December 31, 2008, the loop never terminates. The code checks to see if days is greater than 365 in a while loop. It then handles the greater than 366 condition but never checks if the days is equal to 366.

Write as Little Code as Possible

The less code you write the fewer bugs you will have in it. Most languages today have built in support for common tasks. One example of this is the Java Calendar object which would allow a developer to do what the zune code does using the platform APIs. Unfortunately, High level languages are usually not an option when writing code for a small embedded device.

Tips for writing as little code as possible:

• Start with a high level language. Preferably use an agile and dynamic language if possible (Groovy, Ruby and Python etc). This of course depends largely on the requirements of the application.
• Where possible using the built in platform APIs to do what you need.
• Use open source software to fill in the gaps of platform APIs. Apache and Codehaus are great sources of open source software that are commercial friendly to ship.

For one project I was working on, I needed to get access to the SOAP headers of web service call. I am using the Apache CXF services stack. There were quite a few threads on how to get access to SOAP headers from an interceptor. In my case, I needed the contents of the header inside the implementation.

wsdl2java does have the ability to create a java service class that provides access to the headers using the -exsh option. This was not an option for me since even with this flag on, the headers were not added to the service calls. I think it was how the WSDL I had was designed. This meant that I needed to do the work of pulling out the headers myself.

The SOAP headers can be retrieved from the JAX-WS SOAPMessageContext the easiest. Getting access to this was not trivial. I added the resource annotation to get access to the WebServiceContext. Unfortunately the MessageContext this gave me was not a SOAPMessageContext and provided no way to access the SOAP headers.

After looking through the code for the SoapMessage CXF class, I found how it gets the headers out of the Message. I came up with the following to access the headers from my implementation class:

@Resource
private WebServiceContext context;

private List<Header> getHeaders() {
    MessageContext messageContext = context.getMessageContext();
    if (messageContext == null || !(messageContext instanceof WrappedMessageContext)) {
        return null;
    }

    Message message = ((WrappedMessageContext) messageContext).getWrappedMessage();
    List<Header> headers = CastUtils.cast((List<?>) message.get(Header.HEADER_LIST));
    return headers;
}

This provides all of the headers available. To get the specific one I needed using JAXB I added the following to my code:

List<Header> headers = getHeaders();
if (headers != null) {
    for (Header h: headers) {
        Object o = h.getObject();

        // Unwrap the node using JAXB
        if (o instanceof Node) {
            o = getJaxb().createUnmarshaller().unmarshal((Node) o);
        }

        if (o instanceof DesiredHeaderType) {
            // Do whatever is required with the header object instance
        }
    }
}

This way of accessing the headers turns out to be much simpler than writing an interceptor and trying to stuff the results of that into the request.

This past week was the Electronic Entertainment Expo (E3 for short). Most of the major video game console vendors, publishers and developers get together and show the press some of the new stuff happening this year.

This year Microsoft unveils their new and improved Dashboard. The dashboard is the main Xbox interface that is used to navigate the downloads store, games, achievements, friends and media. Initial reactions on it were mixed. Some people felt it tries to dumb things down.

As a consequence of this new press, Microsoft’s Nelson indicates that a href=”http://xboxfocus.com/news/618-using-new-dashboard-optionable/index.html”>you don’t have to use the new interface. The entire old interface is still present and can be accessed with the press of a button.

There are two fundamental problems I have with how they are approaching this problem. The first is that this is a violation of the DRY principle. Don’t Repeat Yourself. Most of the time we talk about not repeating code sections but I like to apply this to interfaces as well. In most cases, I cannot see a reason to create two interfaces that do the exact same thing. Users can be confused when they are presented with multiple ways to do the same task. From a user’s perspective they want to be instructed how to use the feature correctly and they expect a single answer for this.

The other problem with Microsoft’s approach is that they created multiple ways to do the same thing. I have been working on projects where new interfaces or features were proposed do deal with specific problems. These features were clear improvements over the old way. Inevitably someone asks “Is it possible to keep the old way of doing things as well as the new” or “can’t we have a button to enable the old interface”. These comments should cause you to rethink how good the change is. The new feature always one of the following: better than the old feature, worse than the old feature or neither better or worse than the old feature. If it is better, adopt the better feature. If it is worse or not any better, the feature should be revisited as to why the new approach was taken. Maybe a tweak of the feature could be better. The best option is rarely to keep both ways of doing things.

It is always best to provide a single way to do a single task. This creates a clear and consistent interface for the users. It also creates less confusion for users and makes it clear what is the proper way to use the product.

A few weeks ago my Honda Civic was experiencing an intermittent clunking noise. I am by no means an expert in automobiles but I am fairly certain this noise was not “normal”. I proceeded to let the dealer know about this problem when I had it in for maintenance. I explained how it seems to make this noise sometimes when accelerating or decelerating.

Were they able to fix my problem? Of course not. They were not able to reproduce the problem and consequently did not fix anything. For car problems, we accept this and as a user of the car I am supposed to ignore this until it is either reproducible all the time or something major breaks. They did assure me that they did not see anything major wrong so there was no danger in driving the car.

In the world of software development, this just does not fly. If the user has a problem, they expect it to be fixed regardless of if they can reproduce the problem or even adequately explain it. For example, a support technician brought a customer issue to me where a server product hangs sometimes for no reason. It only happens on one machine and it works fine everywhere else.

The customer wants two things. An explanation as to what is causing the problem and a fix if one is possible. This is not an unreasonable request. I essentially want the same thing for my car. But how can this problem be fixed if it is not reproducible?

There are no coincidences in software. If it happens once, it can happen again. I do not care if the problem was caused by planetary alignment and cosmic rays… the planets can align again and there are always cosmic rays.

Your job as software developer or support technician is to gather as much information as possible to narrow down the problem and to try and find the root of the problem. Find out when it is likely to happen. Try to come up with a reproducible set of steps that will cause it again. With enough work you should be able to figure out what caused the initial issue. Once you find out the cause, then you can fix it like any reproducible issue.

The “customer is always right” is a typical business saying. In some ways the customer is always right because they choose whether to buy your software or not. But are they always right?

I’ve been watching a lot of the television show called House M.D. It is not your typical run of the mill evening drama. It features a doctor called Gregory House who uses his un-conventional style to solve diagnostic medical cases.

“Dr. Gregory House (Hugh Laurie) – Department Head: Department of Diagnostic Medicine. The show’s protagonist, Dr. Gregory House is a maverick diagnostician with a double specialty in infectious disease and nephrology. Dr. House is seemingly lacking in bedside manner and prefers to avoid direct contact with his patients whenever possible. Due to an infarction in his right thigh, House lost a substantial portion of the muscle in his upper leg and must use a cane to assist with walking. As a result, House is also forced to deal with constant physical pain, which he manages through a dependency on the prescription pain medication Vicodin. Although his behavior can border on antisocial or misanthropic, House is viewed as a maverick physician whose unconventional thinking and excellent instincts have afforded him a great deal of respect and an unusual level of tolerance from his colleagues and the medical world.” – http://en.wikipedia.org/wiki/List_of_House_characters

One of the sayings House has is that “everyone lies”. Often he proves this in the show by showing the dark side of seemingly innocent character who tries to hide information from the doctors they feel is irrelevant. Often times that information is just what House needs to solve the case.

I would not go as far as to say that I believe everyone lies but I would say that most people lie. In particular, customers lie. Often the lies are more subtle like in the show. Customers try to protect themselves by hiding information they feel is irrelevant, such as un-related changes to the system that they may not be aware effect your software. They also feed you disinformation based on how they understand your software works.

Every computer savvy user gets frustrated when you call tech support for your DSL line and you have to go through a huge list of steps to prove the problem is exactly what you told them it was. Why do you have to go through this? Because you lie! The support process requires that they can verify you have diagnosed the problem properly. This requires that you follow a long, boring series of baby steps.

So how should we handle customers that lie? We need to realize that they lie and walk them through each step. If they say “your software crashes”, walk them through the process and verify the steps they took to cause the problem. If they say, “nothing has changed in the configuration”, verify this for yourself. Something could have change without them realizing it.

Remember to always be respectful. Often customers lie without knowing it. You need to keep them happy.

The customer is always right but the customer lies.