Droplets of Yes and No

Not only did he help create a kick-ass useful website (flickr), but he knows how to respond to mis-directed emails:

http://valleywag.gawker.com/5288759/flickr-founder-calls-nuked-user-a-dick

The more I used Unit Testing (particularly JUnit) the more I like it. It is a great way of tracking progress in your code, and more importantly making sure you haven’t broken something in the process.

I’m not a huge fan of traditional Test Driven Design (TDD) though. My biggest complaint is writing a battery of tests before writing the actual code feels like putting the cart before the horse. I’ve been experimenting with it, and most of the time I’ve found  that if my code is structured “correctly” (i.e. a well defined API/interfaces, dependency injection, etc.) TDD will work pretty well.

However I have found that I like to use TDD one test case at a time. Basically I will get the basic framework of my class(es) together, and then as I refine the capabilities of the class, add in a few tests to catch one or two conditions. Then I work on my code to make sure that it is performing as expected. Once everything is going well the tests pass and it is time to move on to the  next part of the class.

The big advantage for me in this is that I only have to worry about getting a small number of tests to pass instead of all (or a large number) of them. By breaking the tasks down into smaller pieces I find that I spend less time “dreaming” about how my code would work (and writing tests that don’t accomplish much or have to be re-written as reality sinks in). Instead I’m able to focus on a single problem and solving it.

This is a similar approach as to what is advocated in the unit testing community: When a bug is found, create a test that exposes it. Then fix the code and the test should prove that the underlying problem is gone. I really like that approach and I have begun doing that as often as I can. So far it has really paid off in terms of making sure my code doesn’t have bad case of “but-I-already-fixed-that!” type of bugs.

Question: How much does python rock?

Answer: More and more every day.

Today I was writing (for what seems like the millionth time) a little script to read CSV (Comma Separated Values) file. After running into the same issues over and over (picking a delimiter, escaping delimiters, etc.) I decided my sanity is worth the 30 seconds it would take to see if someone else has already written a CSV library. It turns out python has one built in. Since 2.3. D’oh.

import csv

lines = csv.reader(’myfile.csv’)

That’s all that’s needed to read in a csv file and have it properly handle the delimiters, even when they are inside of escaped text (i.e. something like “$3,000″ will be read as $3000 instead of $3 and 000).

Python rocks again.

About two years ago I finally decided I was tired of my scratched up glasses and that I would get Lasik eye surgery. I was talking about this with some friends recently and thought I should do a post to talk about how things are now that I’m a few years away from it. I did a lot of research on Lasik before my operation, and a lot of what I found talked about the procedure itself and the immediate time after. I thought I would write this blog post and talk about how things are a while after the surgery.

In short: Pretty good!

As a computer programmer I’m rather attached to my eye sight. I was concerned that staring at screens all day might be in for a rough ride because you will have “dry eyes” after the surgery. Some days were tough, but using the preservative-free eye drops (like the doctor suggests) really helped with this. For me the dry eye problem went away pretty quickly, I would say within a month or two I was only having to put the drops in a few times a week as opposed to a few times a day.

The doctors said my eyes were in good shape and it really showed in the recovery phase. The only thing that seems to heal slowly for me was my night vision.

Loss of night vision is a common side effect of Lasik. My night vision has gotten better since the surgery, but it took almost a year, and it still doesn’t feel quite the same as it did pre-surgery. The flip side of this is that in low-light conditions, I feel like I can make out some details a little bit sharper than I could before. I know that sounds odd, but it seems like as long as there’s some good like like a quarter-moon or so, I feel like I can see better than I could with my glasses in that same condition.

So, all in all I’m pretty happy with the way things turned out. Some people do suffer from negative side effects for longer or more intensely than others, but I think that as long as you follow your doctors instructions:

• Moisturize (your doctor will tell you how, usually with preservative free eye drops)
• Don’t rub your eyes!
• Use the medicines and cremes as directed by the doctor.
• Take your vitamins, eat your Wheaties, and get lots of rest.

So that is pretty much my follow up report. I’m glad I did it and I encourage others to talk to their eye doctor if they are thinking about it. I had my surgery at LasikPlus here in Atlanta Ga., and the staff was great and very helpful. Check them out if you are thinking about it!

1. Get the itch to finish a stale project
2. Carve out some time to work on stale project
3. Upgrade your software stack to make sure you are current*
4. Spend hours figuring out something simple isn’t working
5. Curse, debug, repeat
6. Curse more when you realize that step 3 was where the train went off the rails

Yes, I made the mistake to upgrading the Google App Engine Launcher only to discover that for some reason it doesn’t play nice with the Django Helper project (or apparently the latest version of Django).

*sigh*

I know, I can go and continue development without those tools, but I was really looking forward to playing with the AppEngine (using a Django project).

Some days you are the pigeon, some days you are the statue.

There’s nothing worse than having a program that has been running great for X amount of time suddenly start crashing. Especially when you discover that its choking on unicode characters you didn’t expect.

With web addresses (or URLs if you prefer) starting to have unicode values in them, this is a problem that will probably be more common. I noticed this problem when trying to get to web pages that had special characters in them (for example, URLs that feature characters with accent symbols such as in the Spanish language) would cause urllib2.open() to fail. Several website suggested url encoding these special characters, but I didn’t have much luck with it.

Also today I was using Django’s model layer to store some data (web pages) into a database so I could try some things out. Even though Django is unicode friendly, if there’s the slightest problem in mapping text it will pull out the dreaded UnicodeDecodeError at some point.

To get around this I wound up just using unicode(text, errors=’ignore’) to make sure that the text that was being passed in was indeed unicode, and to ignore any errors that cropped up. Not ideal, the better solution would be to map correctly, but better than being stuck with data you can’t load/process.

I’ve been poking around with a little python based web crawler/robot/scrapper that I’ve written. Its a pretty simple script and over time I’ve added little bits and pieces to it in order to try improve, etc. etc. etc.

One thing I added was an ability to track the URL’s I’ve already visited. Trying to be a polite, I decided that this was a good thing to keep from hammering the same web sites over and over needlessly. Basically, I put the address into a dict, and then after a successful visit to the address I set the value of the address to 1. (This way I could collect addresses, but no visit them right away, but always make sure I got back around to visiting them. Or if I needed to visit them multiple times I could keep track of how many times I’d been there.)

Python provides a really simple way of serializing data by using the pickle module. Just open a file, then call the dump() method with the file and the data as parameters, and boom, all done! Going in reverse and reading serialized data is just as simple.

Except…..

What happens if there’s an exception thrown in your code? (Bad URL, unicode that you’re not prepared to handle, network down, etc.) And what happens if this happens to occur while it is trying to dump the data your interested in? Well friends, I can tell you: It ain’t pretty.

Because I had already called file() on the data file, the old data was gone. *poof*. Since the new data was being written out when the new the problem occurred, the new data file was corrupted.

There are several lessons that can be learned from this:

1. If you are going to re-use a program/script more than once, go ahead and build it right.
2. If you are going to write out data make sure it is either
   a) Backed up first (i.e. write to a different file first, then after the file is closed successfully, overwrite the old one)
   b) Written to something like a database which is designed to not get corrupted at the drop of a hat

And as a bonus tip: You should always back up code (and data) that you generate. As I said, this was a quick-and-dirty program at first, but it had obviously grown in it scope and importance. My new metric for “needing to back this up” is if I have checked the code into a repository: If you are willing to check in the code, you need to save the data (config, output data, etc.) as well.

Fortunately I didn’t loose that much or anything that was earth shattering. But all it takes is one time…

So, lets say you are working in XCode 3, and you are trying to build a PyObjC app but for some reason the python code you wrote isn’t getting called.

If you are seeing weird errors in the console when you run the application (specifically “Could not connect the action <your_action> to target of class <your_AppDelgate_class>”), here’s what it probably is: In the AppDelegate python file that XCode creates it uses spaces to do the indentation of the methods.

I hail from a land where tabs are used for such tasks, and the use of spaces is look down upon in much the same way that pick ones nose in public is.

At any rate, what’s happening is when the code is compiled/run the python interpreter is not recognizing the new methods that I put in (using tabs to indent) and instead of giving me a warning about inconsistent indention, it fails somewhat silently in that it groans it can’t hook the action up to the class and that’s it. It wasn’t until I turned on the invisible characters that this occurred to me that it might be the root cause of my problems.

Once setting all of the indentions to be the same, I chose tabs, everything worked like a champ.

By the way, if you are looking for some tutorials for PyObjC in XCode 3 (there are a ton of XCode 2 tutorials out there, but things have changed ever so slightly in 3), be sure to check out these links:

• PyObjC Hello world for XCode 3
• PyObjC Resources

I just finished trying to use XCode 3’s built in support for Subversion. It did not end well.

For starters I created a new XCode project, ran it to make sure it could build, and then decided to check it in to svn so that I would have a “clean” starting point in case I ran into problems. This is how I normally work in other IDE’s, and I’ve never had a problem. Until today.

After setting up the repository in XCode I couldn’t find how to set the project so that XCode would “know” it was under version control (or SCM as XCode refers to it). If you look at the project’s properties, there is a drop down list the lists the available repositories (or it offers to let you set up a new one, which is a nice shortcut). However, the repository I just added is grayed out. After a half hour of trying various “obvious” things I gave up and googled around.

It turns out that XCode can’t create and then check in. The project has to be put into the repository, then checked out before XCode will recognize that it is under source control. Very unusual, but what ever, I decided to play along.  I used XCode’s svn browser (which isn’t half bad) to upload the code, then deleted the project and checked it out.

And right about then is when it became obvious that I was traveling the wrong way on a one way street. Here’s an executive summary of what I found so very, very, very wrong with XCode’s SCM integration:

1. I had to create the project, check it in, then check it out for XCode to realize that it was under version control.
2. Although XCode has an svn repository browser that allows me check in and out the code, it doesn’t seem to know anything about XCode projects as it checked in the build directory without asking. For me (and a lot of developers I know) the last thing I want in version control are files that are going to change constantly (i.e. object files). Checking in a finished version, fine. Checking in a ba-jillion intermediate files: fail.
3. After checking out the project, XCode began complaining about there not being a lock on a file in the build directory. The error message actually said “This might be a bug, please report it.” Sorry Apple, I’m a nice guy, but I’ve got limits. This seems like something that shouldn’t happen on a fresh project (i.e. all boilerplate code, I haven’t done a thing to the project yet).
4. Deciding that I might be slightly smarter than the computer, I decided to delete the build directory in the repository. I tried to refresh the project to let it know there was a change it might need to be aware of. Instead I was made aware of the fact that there doesn’t seem to be one central view of what’s changed for the project. The near-useless SCM menu item in the project reported that one file had change, and I could not seem to get it to even look at the project as a whole. Maybe I’m spoiled by the way Eclipse handles this situation (which provides a bunch of clues in different areas, but all of which lead to a pretty straightforward GUI for dealing with changes). Even TextMate handles this decently (just not as good looking as Eclipse).

By this time I just gave up. I removed the repository connection from the project settings, and decided to just go Command-line for doing my updates/commits. Life’s too short to be chasing my tail on stuff like this.

I haven’t written much lately because I’ve decided to having two ears and one mouth probably means I should spend more time listening rather than talking.

It is an interesting exercise, trying to tune the world in while not adding to the noise. For programming I think this is an essential skill, it is how one learns new things. Lately I have found that having a good cup of hot tea and sitting back to take in things (bugs, new ideas, etc.) really helps me to move forward.

I’ve been hanging out at my favorite independent coffee house Tazza doing some programming work and thinking, and I have to say its a really cool place to hang out. I love the hot tea there, and from what everyone has told me the coffee is really good. The staff (hi everybody!) is very friendly and love to chat. That along with the cakes and cookies combined with the great (and free!) WiFi makes this little coffee bar the ultimate place for me to get away from the grind of the office.

I always wondered why some developers hang out in coffee shops, and now I think I know why: its a good place to be in order to keep your perspective. I might be the only programmer in there, so hearing other people talk about other things helps me to keep things in the right light.

So if you are in Atlanta (especially the Buckhead/Midtown/Georgia Tech area) you owe it to yourself to drop in to Tazza and enjoy a good cup. (The address is 1700 Northside Drive, Atlanta GA 30318 or just use this link)