Jason in a Nutshell

All about programming and whatever else comes to mind

Make sure you use the right method

Posted by Jason Baker on April 29, 2009

Pop quiz:

What is the difference between HTTP POST and HTTP GET?

If your guess is that GET sends data via the query string while POST send data in the request’s body, you are… absolutely correct.  And I hate your guts.  Why is this?  It’s not that what you’re saying is wrong by any stretch of the imagination.

Idem-what?

I hate you because while GET and POST do send data differently, that’s not the biggest difference between the two.  The biggest difference is that POST methods are idempotent.  What does this piece of programmer-ese mean?  How many times have you seen something similar to this Chrome form?

 

confirm form resubmission

confirm form resubmission

On poorly designed websites, you probably see it a lot.  Web development newbies will often ask how to disable this page (or its Firefox equivalent).  There’s a really simple answer for this:  you don’t.  That warning is there for a reason.  You will see that warning every time you hit the back button to go to a page that was requested via HTTP POST.

The reason for this is that POST requests aren’t idempotent.  Idempotent is just a fancy word meaning that something won’t make changes.  As the HTTP 1.1 specification says:

In particular, the convention has been established that the GET and HEAD methods SHOULD NOT have the significance of taking an action other than retrieval. These methods ought to be considered “safe”. This allows user agents to represent other methods, such as POST, PUT and DELETE, in a special way, so that the user is made aware of the fact that a possibly unsafe action is being requested.

Suppose we’re writing software for a web forum, and we have a URI /forum/submit-message.  If we send data to this form, it will post a message.  Now, imagine that after submitting a message, the user visits /forum/view-message and then hits the back button.  They will go back to /forum/submit-message and send the same message over again if you were to use HTTP GET instead of POST.  However, if you were to use POST, that would be prevented by the maligned “Confirm Form Resubmission” page.  Sometimes annoying things can be useful, eh?

What’s the point?

The point is that nothing is more annoying than a website that uses GET and POST incorrectly.  And you don’t want to lose users by using the wrong one, do you?

Posted in Programming | Tagged: , , , , | Leave a Comment »

You need to worry about deployment

Posted by Jason Baker on April 29, 2009

Oftentimes, people used to using PHP or Classic ASP will give up on Python because deploying Python scripts isn’t just a simple matter of copying and pasting files.  Usually, this isn’t because Python is making their lives difficult.  It’s more a matter of not thinking about how to deploy your scripts ahead of time.

Now, I’m far from a highly experience programmer, but I can tell you one thing.  Deployment is a detail that will come back to bite you if you don’t spend a little bit of time on it up front.  So here are a few pointers I’ve come up with after experiences in deploying Python scripts for web apps.

  1. Batch/Shell scripts are your friends.  A common objection is that a person just doesn’t have time to learn a new tool for deploying.  I have two responses to this: 1)  Drop that attitude otherwise you’ll never get anywhere as a programmer and 2) Don’t use them if you really don’t want to!  While batch and shell scripts aren’t the prettiest options, they’re a lot better than having nothing to automate deployment at all.  In fact, for the basic one or two page webapp, you can’t really do much better.
  2. If you invest some time in Continuous Integration, you won’t regret it.  I know what you’re saying.  Continuous Integration is a Java thing.  It’s too complicated.  And you’d be right to a point.  However, I would argue that making sense of the complication is worth your time.  It’s way too easy to deploy something that doesn’t work because somebody forgot to run their unit tests.
  3. Site-wide packages are evil.  If you aren’t already, you should really be taking advantage of virtualenv.  That is, unless of course you enjoy troubleshooting weird ImportErrors because of that egg you installed using the setuptools develop command a month ago and forgot to remove.
  4. Don’t underestimate the value of good docs.  Having good documentation is just one of those things that don’t become obviously necessary until it’s too late.  Don’t leave yourself trying to figure out how that one function you wrote a year ago works.  Write documentation as you go and use a tool like sphinx to turn it into a webpage.  This ties in with point 2.  Using Continuous Integration will make doc generation that much easier.

Admittedly, doing this stuff can be a pain.  And you might get scorn from co-workers for not having something ready the next day.  But it will be worth it.  You’ll be surprised at how much time you’ll save in the long run.

Posted in Programming | Tagged: , , , , , , , , | Leave a Comment »

The magic of python decorators

Posted by Jason Baker on April 25, 2009

Decorators in Python are one of the language’s more “magical” features.  Personally, I’ve tended to glaze over decorators in code because they always seemed to be fairly self-explanatory.  But how do you make your own?  Personally, I think understanding the uses of decorators and being able to write your own is one of the points where a python newbie transitions to a knowledgeable pythonista.

But what is a decorator?

In actuality, there’s not an actual language construct to define decorators.  Any function that takes a function as a parameter and returns one as a result may be used as a decorator.  These are known as “higher-order functions” in functional programming circles.

Chances are, you’ve already seen decorators in use and maybe even used them.  I’ll give you a common example of decorator usage:

class SomeClass(object):
     @property
     def x(self):
         return 5

>>> var = SomeClass()
>>> var.x
5

For those of you familiar with the concept of properties, it should be pretty straight-forward what’s going on here.  But where the heck did property come from?  I’ll give you a hint.  The above code works identically to this code:

class SomeClass(object):
    def x(self):
         return 5

    x = property(x)

 

I can write my own?!

Yes you can.  A lot of well-written libraries make very good use of decorators.  And given the right situation, the little bit of syntactic sugar they provide can do a lot of good.  But decorators aren’t just there for others to define.  Once you wrap your head around decorators, they can save you a lot of copy-and-pasting (which you’re not doing anyway, right?) when used in your own code.  To illustrate this, I want to show you a couple of very much real-world cases that I’ve found decorators to be useful.

Those pesky connection objects

I have a library that needs to call a few particular stored procedures in a SQL Server database.  Because my ORM doesn’t support stored procedures, I have to use straight adodbapi.  The calls look something like this:

import adodbapi

def LookupPerson():
    conn = adodbapi.connect(CONNECTION_STRING)
    try: 
        #do stuff here
    finally:
        conn.close()

This is all well and good for just one function.  But what happens when you need 4 or 5 of these?   And what about the visual cruft that the try finally block is adding to the function (adodbapi doesn’t support with blocks before you ask)?  I’m sure you’ve already guessed the solution by now.  Here’s how you can solve this problem:

import adodbapi

def with_connection(func):
    def _exec(*args, **argd):
        conn = adodbapi.connect(CONNECTION_STRING) 
        try:
             func(conn, *args, **argd)
        finally:
             conn.close
     return _exec 

@with_connection
def LookupPerson(conn):
    #do stuff here

LookupPerson()  #conn argument is passed by the decorator

 I think that the simplification that happens with LookupPerson here should be obvious.  But what is the purpose of the *args and **argd shenanigans?  The documentation covers arbitrary argument lists in depth, so I won’t go into too much detail.  But what happens if I want to lookup a person by name?  The LookupPerson function would be transformed to this:

@with_connection
def LookupPerson(conn, name):
    #do stuff here

LookupPerson('Bob')  #conn argument is passed by the decorator
LookupPerson(name='Jill')

 In fact, I can decorate any function that takes any number of arguments either by keyword or by position.  Pretty neat, eh?

Error handling

When doing web applications, it’s pretty important to have decent error handling.  But setting this up can be a pain.  For instance, what if I wanted to make my django application print a wonderfully informative traceback to a log file?  I could do that like this:

from traceback import format_exc

def index(request):
     try:
          #do stuff
     except:
         logging.log(format_exc())
         return HttpResponseServerError('Error!')

But this definitely can become problematic.  What happens if you duplicate this code in all of your view functions and you want to make a change to your error handling?  The solution is simple:

def handle_errors(func):
     def _handler(*args, **argd):
          try:
               func(*args, **argd)
         except:
                logging.log(format_exc())
                return HttpResponseServerError('Error!')

@handle_errors
def index(request):
     #do stuff here

 As you can tell, this allows us to make our views worry about actually doing stuff instead of constantly handling errors.  Yes, there are also middlewares for doing this kind of thing.  But then I wouldn’t have a reason to make a blog post about python decorators, would I?

Conclusions

Ok, so I’ll admit something.  Python’s decorator syntax is ugly.  Its Java-like syntax alone may even be enough to scare some off.  But as I’ve show, there are at least a few real-world cases where they are useful.

What are some other decorators that you’ve found to be useful?

Posted in Programming | Tagged: , , | 2 Comments »

Software update hell part II

Posted by Jason Baker on April 24, 2009

I just wanted to point out that Kas Thomas has a very excellent blog post on automatic updates that is strangely similar to my own piece Software Update Hell.  As much as I’d like to use this as an opportunity to say “Great minds think alike,” I think this is more a case of software updates just being a total pain in the ass to install on some systems.

Posted in Programming | Tagged: , , , , , , , , | Leave a Comment »

Java vs C# vs Python vs Ruby: an “objective” analysis

Posted by Jason Baker on April 21, 2009

At my place of employment, we’re looking to migrate some of our old Classic ASP applications to something newer (yes, we still have actively maintained Classic ASP code). So my boss asked me to write up an analysis of the different options we have available.

Now before I give you the link, I have a few disclaimers and other random thoughts:

  • I tried to avoid editorializing so that this can be objective as possible, but it’s impossible to discuss these kinds of issues without being subjective.  Thus, don’t take this as the gospel truth.
  • There may be errors here.  In fact, I can almost guarantee that there are errors with Java and Ruby because I’m not terribly familiar with them.  If there are errors, feel free to leave comments.
  • My boss is a Java guy, so I left some blanks that I’m pretty sure he can fill in.
  • Some of these are blatant oversimplifications.  There’s only so much data that you can squeeze into a spreadsheet.
  • I’ll try and keep up with this for a while, but chances are that I won’t for long.  These languages are all being changed.
  • I’m biased towards Python.

Ok, without further ado, here’s the link:

http://spreadsheets.google.com/pub?key=p7efJLoHuYE-iw6JxBmpSQg&hl=en

Posted in Programming | Tagged: , , , , , , , | 12 Comments »

Finding a user’s group membership in Active Directory using Python

Posted by Jason Baker on March 17, 2009

I spent some time trying to figure out what the best way is to determine group membership in Active Directory using Python.  The solution is almost embarassingly easy, but can be difficult to find for someone who’s not familiar with win32 programming.  To make matters worse, google is almost totally unhelpful for this.  So here’s the solution:

import win32net
win32net.NetUserGetGroups('domain_name.com', 'username')

 

To do this, you’ll need pywin32 and a windows computer.  Calling win32net’s NetUserGetGroups function will return a list of tuples.  Each tuple will contain the group name as element 0 and an attributes flag as element 1.  Don’t ask me to elaborate any more on the attributes flag because I can’t.  🙂

Posted in Programming | Tagged: , , , , , , | 1 Comment »

A Case for People-Friendly Computers Part I

Posted by Jason Baker on March 6, 2009

From Toshiba’s Akimu Robotic Research Institute comes a rather funny story.  A robot they had programmed to emulate human emotions developed a bit of a strange problem:

After some limited environmental conditioning, Kenji first demonstrated love by bonding with a a stuffed doll in his enclosure, which he would embrace for hours at a time.

 

What they didn’t count on were the effects of several months of self-iteration within the complex machine-learning code which gave Kenji his initial tenderness. As of last week, Kenji’s love for the doll, and indeed anybody he sets his ‘eyes’ on, is so intense that Dr. Takahashi and his team now fear to show him to outsiders.

The trouble all started when a young female intern began to spend several hours each day with Kenji, testing his systems and loading new software routines. When it came time to leave one evening, however, Kenji refused to let her out of his lab enclosure and used his bulky mechanical body to block her exit and hug her repeatedly. The intern was only able to escape after she had frantically phoned two senior staff members to come and temporarily de-activate Kenji.

You can read the full story here.

Computers aren’t human

This is indeed a comical story, but those of us who are programmers laugh for different reasons from the rest of us:  we can see it happening.  There are two opposite but complementary problems here:

  1. Humans don’t understand computers very well.  Even the smartest of us fall into the trap of thinking that you can make a robot show affection by writing a “hug” procedure.  We forget that we will also need a “stop_hugging” procedure.
  2. Computers don’t understand humans very well.  They don’t understand that when a human programs it to hug things, they mean for there to be limits.

While these things seem obvious to most of you, us computer scientists tend to focus on the second point rather than the first.  In fact, I would argue that most of our efforts in making software user-friendly run contrary to the first point (this is a point I will elaborate more on later in this series).  

There’s a fundamental problem with this approach though.  Computers are wonderful inventions that are capable of so much.  But in terms of being able to understand humans, I don’t expect any big advances in the near future.  And barring a fundamental change in the way computers work, there probably won’t be many advances in the long term either.  On the other hand, I think that human beings are capable of so much more than programmers think they are (and indeed, they’re probably more capable than they themselves think they are).  So why do we keep beating a dead horse?

I’m starting this series of posts so that I can elaborate more on these points.  Some topics that I’d like to cover include:

  • What does it mean for software to be people-friendly?
  • What’s wrong with user-friendliness?
  • What can we do to make software more people-friendly?

And who knows?  I might even write something insightful along the way.

Posted in Programming | Tagged: , , , , | 1 Comment »

Am I a bad programmer?

Posted by Jason Baker on February 27, 2009

Ever wonder why it is that poor programmers always seem to be the ones that are the most confident in their abilities? I’ve always held the opinion that part of what makes programmers good is to sit down and ask themselves “Am I a good programmer?”

According to David Dunning and Justin Kruger, I may be on to something:

People who do things badly, Dr. Dunning has found in studies conducted with a graduate student, Justin Kruger, are usually supremely confident of their abilities — more confident, in fact, than people who do things well.


Unlike their unskilled counterparts, the most able subjects in the study, Dr. Kruger and Dr. Dunning found, were likely tounderestimate their own competence.

In other words, the incompetent don’t realize how incompetent they are. And possibly even more importantly, they don’t realize how competent everyone else is. This seems to explain why they seem to be so unwilling to listen to others’ ideas.

This serves as a reminder of two things:

  1. When you disagree with someone, make sure to spend a bit of time thinking about their point of view. This is common sense, but we all need to be reminded of this sometime.
  2. Next time you find yourself worrying about how poor a job you did on that one class/method/module/entire program, remember to tell yourself that the simple fact that you’re feeling this way is a good sign.

As it turns out, there is a solution for the overconfident but poor programmer. Unfortunately, it’s not an easy one: you have to make them into non-poor programmers. After all, if they can’t become better programmers, why do you employ them in the first place?

Related links:

Posted in Programming | Tagged: , , | Leave a Comment »

Software update hell

Posted by Jason Baker on February 3, 2009

Have you heard of Conficker?  If not, you should read up on it.  It seems that Conficker has turned into one of the largest botnets in the world almost overnight.  As of the most recent data, Conficker has infected 15 million computers (although other studies place that number at about 8-12 million).

And here’s the kicker:  Conficker exploits a vulnerability that was patched by Microsoft almost 4 months ago.  To make matters even worse, an estimated 30% of all windows computers are still vulnerable to this virus.

Who’s at fault? 

If you’re like me, you are probably appalled by the idea that this many users haven’t installed updates recently.  But before you blame this outbreak on idiot end users, answer this question:  how many times have you hit “restart later” instead of installing an update right away?  I do it for almost every update that comes out.  In fact, I’m in “you should really restart your computer now” mode to write this blog post.

Ok, so writing that last sentence made me restart my computer, but you get the idea.

Now, I’m a user who understands the importance of installing updates.  If the process of installing updates is enough to make me put off installing updates, imagine what Joe Internet User must think of the software update process.  Is it any wonder that we still have to tweak webpages to work with IE 6?

I’d like to compare who’s doing things right with who’s doing things wrong when it comes to updates now.

Who gets it right?

To date, I’ve only seen two systems (that I can think of off the top of my head) that handle updates properly:  Google Chrome and Linux.

Chrome

Chrome’s received a lot of flack for installing updates without prompting the user.  But I think that this is undeserved.  To be fair, I can see where this is coming from.  The idea that a piece of software can make changes to itself is somewhat frightening from a privacy point of view.  It can also be frightening from a security point of view.  After all, how do you know that you’re really downloading updates from Google and not some Internet Bad Guy?

While I can see the merits in these points of view, I have to respectfully disagree.  The benefits of having regular updates trumps this concern in both the security and privacy areas.  After all, if you’re not secure, how can you have privacy?  Plus, if the updates don’t significantly affect the end-user experience, how can they make an informed decision about whether or not to install the update to begin with?

What can we learn from Chrome?  Updates that don’t significantly affect the end user experience should be completely transparent to the end user.

For those of you who still dislike the idea of software updating itself automatically, there’s another way of doing things.

Linux

I should mention that I’m talking about my experiences with Debian and Ubuntu Linux.  Your milage may vary.

To me, Linux is more of a server operating system that made it big on the desktop than the other way around (even though it was originally made for the desktop).   With that said, let me point out that Linux as a whole tends to operate in a diametrically opposed fashion to what I posted for Google Chrome.  Linux makes it very difficult to do things that a computer’s administrator doesn’t want you to do.  This means that on a Linux system, you need to approve every update to every piece of software installed on your system.

I’m sure that there are ways around this in most Linux distros.  I just don’t care enough to find it.  Heck, on Debian I don’t even install the update notifier.  I’m more than happy to go into synaptic, click reload and apply updates.  This is a lot more work (relatively speaking) than clicking “restart now” and having everything done for you automagically.  Why am I so willing to do this?  Two reasons:

  1. I almost never have to restart my computer after installing an update in Linux.
  2. I can choose to install updates whenever I want to.
  3. I can install all my updates in one place.

Granted, my way of doing things on Debian probably wouldn’t work for Joe Internet User.  But there’s a lesson to be learned here:  users are willing to do more work to install updates if it means less interruption.

Who gets it wrong?

I’m sure by now, you’ve figured out what my top pick will be for who’s getting it wrong:  Windows.  I’m going to talk mainly about Windows because most of the software that gets it wrong is just a variation on what I’ll be mentioning here, not because I hate Windows.  So what does Windows do wrong?  A few things:

  • You have to restart your computer at the slightest provocation.  In fairness, Microsoft is getting better about this.  But I still find myself restarting my computer more with Windows than any other Operating System.  The problem is that restarting your computer is a fairly expensive task for most users.  You have to save your work, wait a few minutes for your computer to restart,  set everything back up the way you had it before the restart, and then get back into the swing of whatever you were doing.  This can be a good 15 minute chunk of your time.
  • It nags.  I really think this is a good natured attempt on Microsoft’s part to get you to install your updates.  But Microsoft ignores the principle of Psychological Reactance.  In laymen’s terms, this principle means that the more you try to force somebody to do something, the more they will resist changes.  I mean, tell me that the first thought that goes through your head after the third or fourth “you should restart your computer” or “you should install your updates.” My first thought is usually “NO I WILL NOT INSTALL MY UPDATES.”  This has the effect of making me disable the nags, not installing updates.
  • There’s no central place to install updates.  Unlike Linux, I usually have to have several different programs to install updates to my computer.  That means that at some point, I have to make a decision about whether to update or not for just about every program I have installed.
  • It interrupts.  I’d be willing to forgive most of these if it weren’t for one thing:  installing updates in Windows is an interruption.  Users don’t want to worry about installing updates when they’re reading email or writing a blog post or chatting on facebook.

Are you done yet?

If you’re a programmer, you have a responsibility to your users to keep your software secure.  And part of this responsibility is to make updates easy to install for your end users.  How do you plan on making updates painless?

Posted in Programming | Tagged: , , , , , , , , | 3 Comments »

What I like about FOSS (or I don’t like your software and I intend to fix it)

Posted by Jason Baker on January 26, 2009

So I was reading one of Jeff Atwood’s latest posts (I generally try to avoid the obligatory Coding Horror link, but I feel I should give credit where credit is due) and noticed something pretty amazing while reading one of his links.  Someone’s forked Open Office(OOo).  This is a big deal.  Go-oo promises to be a “meritocracy” driven piece of software.

I’m not going to address the issue of whether or not this is a needed step.  Indeed, there have been plenty of people who criticize Sun’s handling of OOo.  But I personally don’t use OOo very much nor am I involved in its development process.  Thus, I’d rather focus on what this means to the Open Source community and even the programming community as a whole.

Dinnerware and open source

For those of you who aren’t familiar with open source terminology, a fork is basically when a developer on an open source project says “I don’t like the way development on this software is proceeding.  I’m just going to make my own version.”  Usually, this causes some hurt feelings on all sides.  But that’s not necessarily always the case.  I’d classify forks into three categories:

The “I hate you fork” – This seems to be the most common kind of fork, and coincidentally is probably the most likely to fail.  Essentially, this happens when one of the developers is fed up with the way a piece of software is developed and feels that it must be taken in a totally different direction.  The classic example of this would probably be emacs vs xemacs.

The “I like you, but I want to take this a different direction” fork.  This fork happens when someone wants to use existing open source software to build an entirely different piece of software.  The best example of this would be debian vs ubuntu.

The “I like you, but can’t you just do x better?”  This is a fork that happens when someone feels that a piece of software they like has one critical flaw that they feel they can fix.  These tend to be the forks that are most likely to succeed, but not in the sense that they’re likely to stick around and get lots of people to download them.  These forks are likely to be integrated back into their source at some point.  The best example of these that I can think of is emacs vs EmacsW32.

Sometimes you have to kill software to love it

In the history of open ource software, Open Office has been one of the biggest players.  I would put it right up there with Linux and Firefox.  But the more time passes, the more OOo loses that distinction.

Let’s put this into perspective.  Imagine if tomorrow, Microsoft Office were to become a stagnant piece of software with a totally fubared development process (I know, some of you will argue that this is already the case.  Just bare with me and pretend that it’s just now happening).  What would happen?  Probably not much.  It would take any competitor years to catch up with where Microsoft is today, and the only option would be to rewrite everything from the ground up.

I mean, could you imagine if one of the devs from the Office team decided to take the source code and make their own version?  That dev would be sued blind.  On the other hand, open source software presents a built-in solution:  the fork.  When a piece of software is broken and needs fixing that its own development team can’t or won’t make, something drastic has to happen or it will die.  And in the case of Open Office, it sounds like something drastic is happening.

So what happens now?

The best thing that Sun can do is pick up on the changes that go-oo makes and do their best to integrate the project.  Unless somebody at Sun completely drops the ball on this (which isn’t impossible mind you), there’s simply no way that they can fail.  OOo has brand recognition that nobody short of Microsoft Office has.  I see one of two things happening here:

  1. Sun completely ignores go-oo and in so doing, makes Open Office obsolete.
  2. Sun is proactive about picking up changes made by go-oo and in so doing makes it obsolete.

Hopefully Sun does what’s best for its software and chooses option 2.  But I’m getting the distinct impression that there’s a high chance of #1 happening.

Posted in Programming | Tagged: , , , , , , , , , , , , , , , | Leave a Comment »