How to Make Developer Certifications That Matter

Last week, I was asked by a potential business partner what I thought of certifications. I immediately clapped my hands like a giddy 3 year old at Christmas and screamed out “It depends, it depends!!!”. Ok, so maybe I didn’t really get giddy, clap my hands or scream, but in standard consultant fashion, I did say that it depends. Then I launched into a spiel about the customers we were specifically talking about who might or might not actually care about certifications and why. This made me think about certifications for developers.

Unfortunately, certifications for developers don’t mean much to other developers. If you have just one certification, people will probably look past it. But if you have a laundry list of developer certifications, other developers are going to start asking what it is that you really know. In general, I’ve found that the number of certifications a developer has is usually inversely proportional to their actual skill. Most people I talk to would agree. But why is this? There’s one simple answer.

The certification system for developers is fundamentally broken.

Certification Is Intended to Signify a Minimum Level of Skill

In some industries, it’s perfectly valid to ask that people go through a certification process. What kind of people you ask? Oh, I don’t know… maybe the guys who run the wiring in your house for example. Need to ground a wire? “Oh, just hook it to a pipe that drains outdoors somewhere. Hey, there’s the pipe from the bathtub. Perfect!”

Meanwhile, he crosses hot and ground, thus giving your wife a good jolt when you turn on the lights in the bathroom because for some inexplicable reason, she showers with the lights off every morning. Don’t ask me how this happened. It’s your wife, not mine. I just write the facts. Anyway…

The intent of a certification program is for an independent party to somehow measure your skill or ability and declare that you are reasonably competent with whatever it is you have been certified for. In the case of an electrician, a board has certified that you aren’t a complete moron and you would probably know the safety and state code violations associated with connecting a hot wire to the pipes instead of the ground. I would probably also want the EMT’s who show up in the ambulance to be certified at what they do, as well as the guys working the nuclear power plant down on the Cape. I’m sure you can think of other places where certifications are probably warranted, but remember, we’re talking about software development certifications here.

Skeptics such as Tom DeMarco argue vehemently against certification of any kind for Software Engineers. Tom makes many good points during his argument about the potential for abuse, de-certification processes and that the people who argue that certifications are for the greater good are the ones who stand to benefit from implementing the system itself.

Touché Tom.

But all of this is beside the point. Tom wrote those words more than a decade ago and our resident “hire the smart guys who get stuff done” leader Joel Spolsky didn’t have much to say about it back in 2000, instead pointing to Tom DeMarco’s original post. Today there are dozens of development certifications out there, many of which either didn’t exist at the time (MCSD) or were still in their infancy (SJCP).

So why do they exist today and what purpose are they supposed to serve?

The Problem of Hiring

Over the years, the IT industry in general has pushed for certifications because managers know how to manage, but not how to identify talent. Most people lack a fundamental understanding of how to interview new programmers and even the ones who do understand it aren’t perfect. Many are constrained by a corporate culture which forces them into accepting one of the 15 people who were brought in for an interview rather than going back to the pool of candidates to keep looking.

So the tech industry giants plodded on, in part at the behest of giant corporations seeking new and “innovative ways” of evaluating developer applicants. An entire ecosystem of software development training was born and guess what?

It sucks.

What’s worse is that everybody knows it sucks.

Certified vs. Competent

Being certified doesn’t mean you are competent. And being competent doesn’t require certification. The original intent of these certifications was to test for competence but thus far, they have failed to do so. Essentially we’re left with a truth table which reads something like this:

Certified	Competent	Reasonable Hire
True	True	True
True	False	False
False	True	True
False	False	False

The only influencing factor in whether the person is a reasonably decent hire is that the person was competent to begin with and certification has done nothing to improve identification of this. In fact, it has arguably made it worse because developers tend to look down on other developers who hold a lot of certifications.

Co-Worker/Colleague Opinions

I’ve never spoken with a developer who had good things to say about any of the major software development certifications. It’s not outright hatred of certifications that I’ve seen; merely indifference. The problem is that a certification doesn’t actually prove anything beyond the fact that you can take an exam and pass it. Big deal.

But what about the money? Isn’t your salary higher if you are certified? Not necessarily. Unless I’m misreading the statistics here, the most recent survey from MCP Magazine shows that the mean salary for an MCP is $77,244 and the mean salary for someone without any Microsoft certifications is $84,839. That’s a $7,595 difference advantage for not being certified. So what gives?

It’s important to note that these statistics don’t give all of the details about how they were put together, so we have to infer some of our data points. First we go back to the reasons to get certified. If the primary drivers behind getting certified are to fast track your career or lend credibility to your abilities because you don’t have a proven track record, then it stands to reason that you are an entry to mid-level programmer.

If the number of certifications is weighted towards those with less experience, they’re going to make less money than those with more years under their belts. This doesn’t mean they make less money. Just less than people with more experience, which tends to make sense. I have yet to see statistics that directly pit the salaries of certified developers against non-certified developers. I’m sure they exist, but I couldn’t find them anywhere for less than a few thousand dollars. I like my readers and all, but it just wasn’t worth the money to find out the details.

How To Fix Developer Certifications

The fundamental problem with developer certifications is that traditionally, they don’t actually prove anything. Microsoft, Sun and all of the other major software vendors have a financial interest in making sure that their technologies are used pervasively in the world. One of the ways to do that is to expand the number of people who are qualified to implement solutions based on that technology. So they lower the bar by producing easier tests and lowering the required test scores, thus allowing thousands of people become certified in an effort to expand their reach, while at the same time ignoring the implications of diluting the meaning of those certifications.

Can this be fixed?

I believe that it can and all it would take is one relatively simple change to make these developer certification exams a lot more meaningful.

Make applicants write code and grade them on two things:

1. how long it takes to complete a moderately complex problem relative to other applicants
2. the number of bugs that are encountered in the final product

That’s it.

You see, the fundamental problem with these exams is that they don’t actually test what developers are supposed to know how to do, which is how to write code. All through college, you take free-form exams for subjects such as Calculus, Differential Equations, Physics, Computer Science, Software Engineering, Senior Projects and others. It’s very rare to find multiple choice exams here and the ones that do exist are so difficult that if you didn’t know what you were doing, you would get the answers wrong anyway.

Some professional exams do this well. For example the Professional Engineer Exam is a two exam certification process. The exam part contains both free form and multiple choice questions. After several months you get your results. Assuming you pass, you must continue to work in the field, provide proof of that field experience and take a much more difficult exam which consists of advanced engineering topics.

There are certain types of financial accounting exams which are notoriously difficult because they use a ladder system where at each stage up the ladder, only a limited number of people are allowed to pass. You must beat the scores of your fellow test takers, rather than beat a predetermined test score. As you progress up the ladder, the competition makes the exams more and more challenging, thus allowing only the best of the best to advance.

Using a developer certification exam with the above criteria would easily accomplish the goal of separating the mediocre programmers from the rock stars. In addition, the higher up the ladder you go, the easier it is to quantify how you relate to the rest of the developers who took the exams.

There are of course some logistical problems that would need to be dealt with. For example, most developers search the internet for code examples pretty extensively. You would need to allow searches, yet disallow chats with other people who might help someone cheat. Doing this for tech savvy developers is going to be difficult, at best. Also, developers would be much more comfortable with a configuration and software setup that mimics their own development environment. These and other problems would need to be addressed.

But where there’s a will, there’s a way and I have no doubt that if a major software vendor were so inclined to do this, it would work. This would put solid metrics on the certification exams and provide meaningful, quantitative measures in place for measuring the development ability of a candidate.

Do you agree or disagree? Please post your comments below.