Entries Tagged 'Software Development' ↓

Have you ever had something on the tip of your tongue, but you just couldn’t get the words out? I felt that way with a Python program I’ve been knocking around with for a while now (its an RPG that I swear I’m going to finish some day…). Every time I looked at the code I kept thinking that there must be a way to make it more data driven (like in the book Game Programming with Python), but I just kept hitting sticking points.

The other day on the Pygame Mailing list, I saw this post describing a simple, elegant, data driven solution for an RPG. I was floored. This code helped me bridge a couple of different ideas together. I had read about making items flexible by making them via “instance relationships” (i.e. damage objects can inflict damage onto breakable objects). I thought it was a really interesting idea, but hadn’t really thought much about how to bring it to life.

At any rate, this chunk of code was a real inspiration and I thought I would share it in case anyone else is curious about integrating data driven concepts into their code. Check it out!

I saw this article this morning:Linux kernel ‘getting buggier,’ leader says | CNET News.com, and I have to say that I’m not surprised.

Its not that Linux is buggy by default, rather that it has grown so much that this was bound to happen. I think that anytime you have a project with multiple developers over a long period of time, and a growing user base you are just bound to get (or find) more bugs.

The statement by Andrew Morton about the difficulty of getting coders to look at old bugs is very true. I experienced that myself here recently. Its hard to go back and re-visit something that once was working (or so you thought) but is now broken in some weird convoluted way.

I wonder if anyone is running any type of code analysis tools over the kernel codebase. I’m sure someone is, its just too high profile of a project for someone to not do that. I imagine that report must be a mile and a half long…

Lately I’ve been trying to work more of a testing into the code I’m writing. I’ve read a lot about Test Driven Development, and while I think it is an interesting approach, it just seems like such overkill in many situations.

For me I’ve found a happy medium by approaching my development in the usual ways (design, then code), but I’ve been trying to inject testing code at the earliest opportunity I can. I’m finding that the more I do this the better my code looks.

So far the biggest difference I’ve seen is that I’m tending to write cleaner interfaces for my classes. This is because with the tests I’m trying to make sure the logic in the class is functioning correctly and sometimes it is nice to see what is going on in the middle of the process. A clean interface makes this task much easier, and as a bonus the cleaner interface allows the code to fit better into the rest of the code base because it is now there with a purpose instead of just being shoehorned into place. (Of course some people will point out that a good design process would eliminate this problem in the first place, but sometimes you just can’t get that level of detail in the beginning. Viva la refactoring!)

The biggest benefit of unit testing is supposed to be finding bugs. While I do find bugs early with unit testing, I don’t think this is the biggest benefit I’m going to see. For me the biggest (and most unexpected) benefit so far has been discovering area where I can decouple (via better interfaces) my classes from each other. I’ve already seen one or two spots where during a refactoring I was able to get a lot of major work done quickly and successfully, and it is because the code was properly coupled with the rest of my framework. That was pretty cool.

So, in short: Don’t be afraid to add some testing code to your project. You don’t have to go overboard with it, even a little bit can help out a lot.

Although we live in the future, and in the future everyone uses XML, there are still times when you need to interface with something that uses fixed position strings to relay information. This happened to me the other day at work and I thought I would share somethings I learned.

My first thought was to find something that would allow me to insert characters into a String at a specified point. Strings are immutable, so they aren’t ideal. The StringBuffer class is designed to be manipluated, so it seemed like the perfect fit. Looking through the API docs I saw that it had useful sounding methods like “insert()”.

Boy was I in for a shock.

When you build a StringBuffer, you can specify the length of it. Since I’m going to be building a String of a specific length it seemed like a snap, just build a StringBuffer of that length. But when I tried to use that StringBuffer, I kept getting index exceptions.

It turns out that when you pass in a number to the constructor it allocates the space (plus 16), but doesn’t put anything in it. For some reason when I started putting strings into the StringBuffer (via the insert() method) I would eventually get the error. Stepping through the debugger showed that the buffer was allocated, but something just didn’t seem right. When ever I would try to print out the StringBuffer it never seemed right, like it was stopping in the middle for some reason.

So, as a quick test I used the constructor that allows you to pass in a String, and I passed in a String of spaces that was the right length. Lo-and-behold everything started working correctly. That was slightly annoying to me because from everything I read it seemed like I shouldn’t have to pass in a string of spaces in order for this to work.

As as side note, I really really really don’t like variables that are just strings of spaces that have to be a certain length. It is almost impossible to count out the spaces, plus if someone ever accidentally removes/adds a space it is a pain to debug. It is much better to specify a length and then build the string of spaces at runtime (if it simply must be done).

So, once I had that figured out I thought it was going to be clear sailing. Ugly, but clear. Wrong again. It turns out the insert() isn’t the best choice for inserting things. Insert doesn’t overwrite anything in the buffer (which is cool, but now that I have my crazy String of spaces I would prefer it to overwrite, but hey this ain’t Burger King so I’m not gonna get it my way). Instead it simply inserts. Which eventually causes the index exception. D’oh!!!!

It turns out that replace() is a much better choice. However it requires a lot of parameter to be passed in, and soon the code I was writing looked like a war zone. So at this point I had to create a new Class to wrap up all of the ugliness and still keep it useful.

It turns out this was a pretty good move to make, because a little while later I discovered that there were times were I would want to insert something, but have it justified one way or the other. Simply adding a method to my new class allowed me to keep the code nice a clean, and debuggable (because all of that logic was contained in once place). Score one for Object-Orientation.

Anyways, the moral of this story is that the Java API is pretty good, but is lacking in some areas (don’t get me started on the nightmare that is the StringTokenizer class). A well crafted wrapper class can help create the functionality that might be missing and also allow you the chance to keep your main business logic code looking clean. Plus the benefit of having that kind of code (the kind that is subject to changing requirements) in one place for debugging and maintenance can not be understated.

A few weeks back there was a very interesting article called Data Mining 101: Finding Subversives with Amazon Wishlists that basically showed how to do simple data mining using Amazon wishlists as a data source. A lot of people saw this article (I won’t say read for a certain reason) and started talking/blogging about how shocking it was that all of this data just out there for anyone to harvest.

This of course misses a big point of the article, that the source of the data isn’t that important, it more about what you can do with that info. A lot of people missed that point, but hey, that’s ok. (Honestly, anyone who was upset that their wishlists can be viewed by the public at large really shouldn’t be posting that information on the net in the first place, no?)

For me this article was amazing because it showed data mining using simple, readily available tools. No fancy databases, no complicated AI programs, just simple script files and the file system. That’s the real beauty of the article the fact that it can be done by anyone (ok, anyone not running Windows). Simplicity really is the key, ya know?

I’m trying to be a better person by wrapping my brain around the Lisp programming language. I’ve got to say, remove-if and remove-if-not are two of the most confusing function names I have ever come across. I’m sure with time and experince I’ll change my view, but as a quasi-Lisp-newbie my mind warps when I read those function names in sample code.

I’ve been debugging a lot of different code lately, and it has helped me remember the importance of logging in an application. Especially when you application is going to be used in the real world. You can plan, and code, and try to account for every situation that might happens, but eventually something you didn’t plan on will hit, and if you have logging, you can use those message to trace down the cause of your problems.

However, it isn’t enough just to be able to say “Yeah, my app has logging”. You have to make sure you have logging in the right places, and that it is reporting the right information. Without those two conditions being met your logging can be next to worthless. The worst thing that can happen is for your logs to be ignored because they don’t give useful information.

These are my suggestions for making sure your application has effective logging.

1. Put logging into your application - Yes, this is very obvious but it is something that needs to be said. An application that dies without telling you anything doesn’t really help you very much does it? Put logging messages in it! Even if you have to resort to just printing out messages to the command line, this is better than nothing. There are plenty of good logging solutions out there (like Log4J), check them out and try to integrate them into your application.
2. Put logging messages in the right locations - The number one problem that developers seem to have with using logging messages is that they print out too much. Most of the time this is because the logging messages aren’t put in the most effective locations. What is an effective location? Any place where you find yourself setting a break point is a good rule of thumb. Other good spots include:
   +Catch Blocks (because its nice to find out quickly what caused that initial exception)
   +Business Logic Layer (errors in this layer are hard to pin down)
   +Any spot that has external I/O (disk reads, network calls, these can all fail for many reasons)
   +Sections of complicated logic (if its hard to understand when you are writing, imagine trying to understand it when you angry customer is breathing down your neck to get the system back up and running)
3. Make your log statements count - You’ve got to get a message out, make sure you communicate the right things. If you app runs in a multi-user environment (like a web app), then maybe the log message should include the user’s name so you can track down and monitor a particular user’s situation. Where in the code was the message generated? (Some logging solutions like Log4J take care of this for you.) If you had to have this message read to you over the phone at 2 in the morning, what info would help you diagnose the problem the fastest?
4. Log consistently - Once you start logging, make sure you log everywhere. If you and your support staff get used to good logging from one module, they are going to assume all modules have equally good logging. Don’t disappoint them! Logging statements should be uniform both in format and in usage through out the application. Also, make sure your logging statements are all being written to a single location. The more places you have to hunt to find a message, the less likely you are to go find it.

As I said earlier, there are many logging solutions available no matter which language/editor/environment you are coding in. A good spot to check out is the Apache Logging project. Another bit of advice is to become fluent in tools like grep. Log files can get large, but grep and other tools can help cut the job down to a manageable size.

Most importantly, remember that while it is a pain up front sometimes, the logging messages will pay for themselves when things go wrong. Notice that I said “when” and not “if”.

So get your logging in place!

Ned Batchelder’s site is always a good read, he’s very insightful about the art of programming. When I saw his posting that Joel Spolsky is a crotchety old man, I couldn’t wait to see what his thoughts where.

When I wrote about Joel’s article I was feeling a little bit down, Joel pointed out something that I knew deep down, namely that programmers today (i.e. me) aren’t quite the same as programmers from yesteryear. Java and other “layers of abstraction” have taken us away to a weird place. Its almost like you don’t have to think as hard to be a programmer today.

Ned pointed out that this isn’t 100% true, its more like you don’t have to think the way you used to. He points out that assembly language and a good knowledge of the inner workings of the computer were necessary to perform the job role of a programmer. These days, a lot of schools aren’t really teaching assembly, and using it in a day-to-day job is becoming rarer and rarer.

Is this “loss of knowledge” really a sign of how bad things have become? Not necessarily. A lot of systems and languages these days go to great lengths to hide the implementation details of the underlying hardware from the developer. Java springs to mind as an example of this. A Java developer doesn’t need to understand what the value in the IP register is.

BUT… if the Java developer does understand how a computer works at a low level, then they probably would have a better insight into performance issues, threading, and other dark corners of computer science.

After thinking about both articles I’m starting to think that while they both have a good point, Ned is really onto something. Perhaps what the computer scientists/developers/programmers of today are missing is the understanding of the computing machinery. I know when I recently took a computer architecture class, I found it very enlightening and I haven’t really looked at any programming language the same since.

Today I decided to do something about a project that has been nagging me. I started write a game last year (or was it 2004? It was started a while ago at any rate…), but then I just stopped working on it for a variety of reasons.

One of the biggest reason was that my design, while it sounded good in theory, wasn’t working out too well in practice. My idea was to make a Wasteland style RPG type of game that used a state machine to control the game. The idea worked pretty well in the beginning. I found a project that was similar to what I wanted to do, then adapted the code and put a state engine in there to drive everything.

It worked well enough until I wanted to add in more features like events, changing maps, etc. Then the code revealed how convoluted it was. A lot if it is my fault, I’m still not in the “Python frame of mind” for everything I do, and that has made things more difficult. I checked around on the internet to see if anyone else was tackling a project like this and lo-and-behold there was. This site has a great little tutorial that pretty much covers everything I wanted to do, but done without an (explicit) state machine.

Instead it uses a different pattern, the mediator. The mediator looks a lot like the observer pattern, at this point I’m not sure what the major difference between them is. At any rate, I had some time today so I set out to refactor my little game and see if I could drop in the mediator/observer pattern in the place of the state machine.

Much to my surprise and delight, it was fairly easy to do. After about 3 hours or so of work I’m now almost at the same point (functionality-wise) that the original code was. At first glance it looks like I’ve streamlined my code base a lot and can eliminate a lot of extra code that was introduced to the system. The only downside I can see so far is it looks like the mediator/observer based code is inefficient in that every game object gets called every game tick. At this point I’ve got a pretty small number of objects, but the moment I get NPC’s and inventory items working I’m going to have to do something because that will introduce a ton of objects to the game world. Truthfully I don’t think it will be too terribly hard, but it is something to think about (the web page mentions this problem and has a few suggestions).

The biggest and nicest advantage of this refactoring has to be the separation that has been gained. Before I was using global objects to try and keep everything in touch with everything else. Now each game object is its own little island, knowing about only those objects it needs to know about. This is really exciting for me, before I was dreading adding new things to the game world because of the work that was involved in attaching them to everything.

Well, back to the salt mines, let me see if I can make some more progress!

Every now and then when I’m feeling nice and confident about whatever it is I’m doing, something will come along and knock me down a peg or two.

Lately I’ve been feeling really good about my development skills. I’ve been thinking a lot about things like refactoring, architecture, reusability, maintainability. I feel that while I’m not a master yet, I think I am well on my way to honing my Jedi skills in the area of software development.

That is until today. Today Joel published an essay about how the schools these days are focusing in on the wrong things (pushing Java and not other languages that require more thinking) and basically not pushing the students hard enough. The article is here:The Perils of JavaSchools

After reading the article I was pretty depressed. I thought I was doing ok, but now I think I need to make sure I master the art of thinking, not just the art of implementing whatever Sun (or whoever) is spouting about this week.

Like Bea, if I am to survive the cruel tutelage of Pai Mei, I will simply have to buckle down and do it. Nothing worth having comes easy, and having the knowledge (and the power that comes with it) that Joel speaks of is something worth having.

It looks like it is time to break out the Little Lisper and begin the training…