In the light of yet another load of lost disks, and peoples' feedback about the lack of expertise within the UK government when it comes to data processing, here's Joel Spolsky on what it's like writing in-house software. (Update: given comments below, and Spolsky's own content, "bespoke" might be a better word, because this is custom software. So where you see "in-house" below, think "bespoke".)
The entire piece is one of three on his site, from a talk he gave to his alma mater, Yale University's computer science department. They're all worth reading for themselves. In case you're thinking "who?", Spolsky has the reputation of being the guy who had got the lowest-ever F-word count in an internal meeting with Bill Gates (back when Bill made Gordon Ramsay look a bit tame), and wrote a piece for us recently about how Google can't rest on its Ajax-y laurels.
Now, here's what he has to say on in-house software. All you in-house programmers out there, does this chime?
New York was the first place I got to see what most computer programmers do for a living. It’s this scary thing called "in house software". It’s terrifying. You never want to do in house software. You’re a programmer for a big corporation that makes, oh, I don’t know, aluminum cans, and there’s nothing quite available off the shelf which does the exact kind of aluminum can processing that they need, so they have these in-house programmers, or they hire companies like Accenture and IBM to send them overpriced programmers, to write this software. And there are two reasons this is so frightening: one, because it’s not a very fulfilling career if you’re a programmer, for a list of reasons which I’ll enumerate in a moment, but two, it’s frightening because this is what probably 80% of programming jobs are like, and if you’re not very, very careful when you graduate, you might find yourself working on in-house software, by accident, and let me tell you, it can drain the life out of you.
Click through for the rest...
OK, so, why does it suck to be an in house programmer.
Number one. You never get to do things the right way. You always have to do things the expedient way. It costs so much money to hire these programmers — typically a company like Accenture or IBM would charge $300 an hour for the services of some recent Yale PoliSci [Political Science] grad who took a 6 week course in dot net programming, and who is earning $47,000 a year and hoping that it’ll provide enough experience to get into business school — anyway, it costs so much to hire these programmers that you’re not going to allowed to build things with Ruby on Rails no matter how cool Ruby is and no matter how spiffy the Ajax is going to be.
You’re going into Visual Studio, you’re going to click on the wizard, you’re going to drag the little Grid control onto the page, you’re going to hook it up to the database, and presto, you’re done. It’s good enough. Get out of there and onto the next thing.
That’s the second reason these jobs suck: as soon as your program gets good enough, you have to stop working on it. Once the core functionality is there, the main problem is solved, there is absolutely no return-on-investment, no business reason to make the software any better. So all of these in house programs look like a dog’s breakfast: because it’s just not worth a penny to make them look nice. Forget any pride in workmanship or craftsmanship you learned in [your computer science course]. You’re going to churn out embarrassing junk, and then, you’re going to rush off to patch up last year’s embarrassing junk which is starting to break down because it wasn’t done right in the first place, twenty-seven years of that and you get a gold watch. Oh, and they don’t give gold watches any more. 27 years and you get carpal tunnel syndrome. Now, at a product company, for example, if you’re a software developer working on a software product or even an online product like Google or Facebook, the better you make the product, the better it sells. The key point about in-house development is that once it’s "good enough," you stop.
So - is that your experience? And could that be behind the problems at HMRC and the Driver Licensing Agency?
Comments
Please note: In order to post a comment you need to be registered and signed in for Guardian Unlimited blogs.
You can register here.
Im guessing its more because Accenture and Crapita are run by conmen myself.
@mawbags: *coughs* you mean in *your opinion*.
And Spolsky is writing about his experiences in the US (note the "New York" bit he talks about? It's not the new part of York), but one senses that it's probably much like that in the UK too. Perhaps moreso.
I get the feeling that Mr. Spolsky hasn't quite cottoned onto the nature of 'work' and that for a lot of people, computing is one of those weird and wonderful 'job' things that normal people have to do.
You could do worse things than work for 27 years in one place, making your colleagues and customers happy. As for 'working in a product company- the better you make the product, the better it sells...' Hmmm? So people only ever buy the 'best made' products, do they?
Plus he calls his blog a 'pulpit'. Urgh!
Yes, isn't it sad that 95% of programmers choose to work on boring old commercial applications like banking and insurance. Hey, get a life guys, and go and join a groovy internet startup outfit where your 'creativity' will be appreciated and indulged. Comparisons with mega-successes like Google and Facebook are absurd. Here, the software IS the product, and its appearance and functionality have to be constantly reviewed. In the real world, the important criteria for computer systems are considered to be things like reliability and data integrity (you know - the things that make your credit card work online). Coolness is (rightly) not a high priority.
Blimey, you guys are a bit hard on him. Seems to me he makes a perfectly good point, which tallies with my experience. There is very little obvious benefit in spending money on making in house software work well. Thus it will be bad, and probably have all sorts of security problems and/or break in an awful fashion, and/or be so badly designed it adds thousands of man hours to a task which could be done better. The powers that be probably won't notice that though. Software for public consumption has to be good, or it won't sell. QED, and fair point.
Moreover, if you have 15 years of legacy software, you tinker and fix it, rather than rewrite it from the bottom up. Major banks have more like 30 years of legacy software, which has more and more software built on top, and nobody really understands what is going on at the bottom level any more. The different iterations of the software are not properly documented (short term cost:benefit ratio again), written in different languages, etc. It led to a disastrous crash with Scotiabank in Canada a year or two ago (I think - Google it).
See http://thedailywtf.com/ for examples of what goes on with cheap harrassed programmers in companies everywhere.
If the "finish" on website is good and the finish on in-house software is bad - why not delvier all in house software through a web portal? That way the finish will have already been decided - it'll be the one that the Cheif Execs daughter thought was prettiest - and it will provide a common and predictable interface for every bit of software.
If the "finish" on website is good and the finish on in-house software is bad - why not delvier all in house software through a web portal? That way the finish will have already been decided - it'll be the one that the Cheif Execs daughter thought was prettiest - and it will provide a common and predictable interface for every bit of software. Thus improving usability.
"[Developing in-house software] can drain the life out of you"
One can also say that developing a shrink-wrapped product and getting a version 1.0 out the door certainly has the same effect.
I like in-house software. Get it to work fine, then move on to something else...
Let's see if we can get that "F-word count" up then, shall we? (I'll put it through he BSG translator, in case the GU censors are switched on today).
>> In the light of yet another load of lost disks...
Go on, Charles; I give in. What exactly does programming have to do with lost disks? I've been a programmer for the best part of 15 years. Do you want to take a shot at guessing how many backup disks, tapes, CDs or DVDs that I've lost in that time? I'll give you a clue: it's the same number that I've ever been responsible for; which is the same number that any programmer anywhere has *ever* been responsible for. (It might help to picture a Polo mint when you're guessing that number).
Moving on to Mr Spolsky's commments...
>> "why does it suck to be an in house programmer.....
>> It costs so much money to hire these programmers --
>> typically a company like Accenture or IBM would charge
>> $300 an hour for the services of some"
Watch out! Massive clue approching!
When you hire somebody from IBM or Andersen Consulting (sorry, that's Accenture now), they are not frakking "in house programmers"!!! They are consultants! (You know what a consultant is, don't you? It's what somebody writes on their CV when they don't want to say "prostitute", which is a far more accurate description of what they do for a living). A real "in house programmer" is a permanent employee of the company, and he (or she) most definitely does *not* get paid "$300 an hour". If he's lucky, he might get paid one third of that amount. As an added benefit though, he'll get to spend extra (unpaid) hours at work, explaining to the $300 per hour contractor exactly how things work at that company, and then watch the bastard swan off into the sunset, having contributed absolutely frak-all to the company's well being.
>> "as soon as your program gets good enough, you have to
>> stop working on it"
Sorry? Where does it say that? I've been a programmer for the best part of 15 years (did I say mention already?) Take a guess at home many times I've heard somebody - either an IT manager, or somebody from the business say "that program's just good enough now; don't touch it again". (Still got some Polo mints left?) And of course, there's no way on God's green Earth that it will ever occur to me, as the original programmer, to think of ways to improve it, is there? Because I'm just coutning the hours until the phone call from Google comes ... or something. (I kind of gave up on the article around here).
Seriously, have any of you frakking journalists ever worked in IT? I mean, actually worked in an IT department, doing any kind of IT job? Do you really have a clue what goes on there?
Cheers,
- Mike
"You know what a consultant is, don't you? It's what somebody writes on their CV when they don't want to say "prostitute", which is a far more accurate description of what they do for a living."
Now now. That shade of green is most unbecoming on you.
@Brownieboy:
>>
>> In the light of yet another load of lost disks...
Go on, Charles; I give in. What exactly does programming have to do with lost disks?
Because the question of in-house versus outsourced versus who does what is what has got the government into the position where disks get misplaced (and created, come to that). Lack of programming skills inside government departments - its outsourcing - leads to the situation where the HMRC is charged £5,000 to strip out data that you'd think could be done in-house.
>>
I've been a programmer for the best part of 15 years.
>>
That all?
>>
Do you want to take a shot at guessing how many backup disks, tapes, CDs or DVDs that I've lost in that time? I'll give you a clue: it's the same number that I've ever been responsible for
>>
You're not giving a good picture of your CV, now, but we'll let that pass.
>>
A real "in house programmer" is a permanent employee of the company, and he (or she) most definitely does *not* get paid "$300 an hour". If he's lucky, he might get paid one third of that amount. As an added benefit though, he'll get to spend extra (unpaid) hours at work, explaining to the $300 per hour contractor exactly how things work at that company, and then watch the bastard swan off into the sunset, having contributed absolutely frak-all to the company's well being.
>>
It still all turns into "in-house" software. (You could call it bespoke, and then I think everyone would be happy. Actually, where you've seen "in-house" anywhere above, put "bespoke". Re-read. Nod in agreement.)
>>
Take a guess at home many times I've heard somebody - either an IT manager, or somebody from the business say "that program's just good enough now; don't touch it again".
>>
You just have to be more assertive in your annual reviews. They'll eventually realise that you have managerial skills, I'm sure.
>>
Seriously, have any of you frakking journalists ever worked in IT? I mean, actually worked in an IT department, doing any kind of IT job? Do you really have a clue what goes on there?
>>
Yes, I've worked in an IT department. I've worked in IT consultancy (of sorts). I've worked in the civil service. I wrote a bespoke program (see what I did there?) in-house to see what the hell a few hundred people did with their time. Turned out it was filling in forms telling their bosses what they did with their time. A key thing in that project was just getting it finished so they could do the consultation - so "good enough" and "don't touch it any more" were two phrases I did hear. Just an experience thing, I guess.
Actually the original comment about con men is pretty much to the point. Not that women aren't involved too, but the transaction is all about confidence. Whether things work or not is rarely a black & white issue in the complex environments that corporate computing generally seeks to manage. It takes a lot of people to get things done; people require management; management requires social skills, which include the ability to raise and lower expectations simultaneously across the variety of personnel -- clients, workers, managers & so on.
Ah, applications programmers, a whole different world. You're stuck with them whether you're writing a complete applications suite from the ground up or writing a few macros because despite what the salesmen tell you application generators don't produce instant results from the touch of a marketing person's mouse. Indeed, one of the reasons why you're getting into this kind of mess is that you're trying to short circuit the design process by using those so-called Rapid Applications Development toolsets. Programming isn't about coding, its about design and testing (but you'd never guess it from the behavior of most managers -- if you're not in that cube pounding the stuff out then you're not working).
I've been fortunate to have spent most of my career as an embedded programmer. There are all sorts of advantages to this -- you're mostly ignored by marketing types, the sorts that like to add functionality and tweak user interfaces, you're far more connected with the system design and you get real-time feedback about whether your creation works or not. It keeps you honest.
HMRC and DLA don't have in house programmers - they outsource it all to consultancies like Accenture. They hire 21 year olds fresh out of university, give them 6 weeks of Java training, and then get them to write the software.
That's why it's so expensive and so bad.
The lesson may be that, while in-house sucks, outsourced sucks worser.
Please note: In order to post a comment you need to be registered and signed in for Guardian Unlimited blogs.
You can register here.