In defence of the lone coder archetype

All too often these days we see companies and methodology evangelists touting the system du jour whether it be Extreme Programming (XP/Pair Programming), or Agile as being the "One True Path" to coding enlightenment. All other be damned is roughly what said mantra translates to in common speak. We're told to beware the engineer who works alone in a dark back room for weeks on end. We're told to such a thing is heresy against the gods of modern application development.

We're being lied to by the methodology evangelists. I'm not here to down speak any of the other methodologies currently in use for software development. I am here, however to point out the benefits of the 'lone engineer' (not to be confused with the lone gunman). Lets set a few things straight though.

A coder with little to no professional experience in the field is not the optimal individual for this kind of setup. This is more for a seasoned professional who has at least been a team lead on several projects from conception through maintenance phases. I'm not saying it isn't possible to pull it off without significant experience, but I wouldn't place bets on such an individual succeeding at the whole task at hand in said company.

This kind of environment requires a disciplined engineer. We're talking about someone who knows the questions to ask, and how to dig deep to find the true needs of the client(s). This also means that the engineer needs to stand his/her ground when it comes to setting a solid schedule for phases of the project(s). There is room for variability in terms of time frames for each phase, but that the plan must be laid out in a linear fashion. This is engineering after all, and one doesn't plan to build the rooms before the foundation is laid.

Agile methodologies are more akin (though not parallel to) modular housing construction. Individual sections or units can be build simultaneous and changed (to a certain extent) by separate sub-teams. Whereas the lone engineer mentality is more akin to traditional construction in which the design is finalised, the foundation is laid and construction occurs in layers from the bottom up with customisations being generally last.

This isn't to imply that software built in this manner cannot change or adapt in a timely manner, far from it. Seasoned engineers/developers understand that the only true constant is change. We constantly work on new ways in which to design flexibility and mutability into our systems regardless of the specifics. This however just goes back to one of my earlier points in that this requires someone who isn't wet behind their ears. You can only know and plan for the unexpected via flexible designs after having cut your teeth in real world projects, and by faltering. Everyone makes mistakes, and it is through these mistakes that we grow and better ourselves. I'm not implying that senior level engineers don't make mistakes, I'm just pointing out that their success to failure ratio is pretty strong in favour of the success side.

One of the key benefits of the lone engineer methodology is that focus can remain solid with no deviation due to intra-coder/intra-team conflicts. And while this can at teams be a negative, overall it proves itself beneficial towards the ultimate goal, a finished software package/eco-system designed from the ground up to benefit the requisitioner(s)/client(s).

The issues of coding standards still must be kept, however there is definite consistency when only one individual is involved. This is also a liability as said engineer must have strict standards, commentary and documentation from beginning to end so that in case of any situation rending the engineer on the project incapacitated in any form, another could come in and continue with minimal delay. These are issues which affect other methodologies, some more than others.

There are downsides as well though. There are no direct peers upon which one can depend in times of need or collaboration. There are no peers to assist in code review sessions, meaning that the QA of any project must be proficient at testing, and enforce strict recursion testing when any changes are made to a product in production as QA becomes that final line of defence against human error. The reality is, no usable (as in serves a real purpose) piece of software is bug free and to think otherwise is the sign of a fool. What is realistic is having a system robust to handle errors when they occur, even if that is infinitesimally infrequent.

I'm getting off track here. My point is that in this newer, more extreme era in which coding fundamentalists believe they possess the one-true-way of coding and all others be damned, we must step back and realise the grave miscalculation they are making so boldly. After all, those methodologies weren't what were utilised to get us where we are today, nor were they even thought of when some of the staples of our art were invented. This doesn't mean that the next great piece of software or language won't be produced in such a manner, as it/they may. It means that we as professionals shouldn't be so narrow minded with tunnel vision when approaching different methodologies in an attempt to find which works best for the project/company/industry at hand.

Why projects fail, or more appropriately why QA cannot be an afterthought in the software cycle.

Over the years I have been in a multitude of software development environments, and bar none, the biggest reason as to why software projects fail can be overwhelmingly attributed to quality assurance.  More precisely it is either the complete and utter lack of a QA process (let alone a team), or simply the absence of sufficient testing procedures. 

Quality of software isn’t simply the fit, finish and packaging.  It is a whole encompassing methodology through which the program(s) involved are designed, revised, (when the time comes) patched, and upgraded.  It has been a long held observation of mine that coders in general don’t think of QA for several reasons.

    ◊    They feel that their programs were well designed and that they’ve accounted for all scenarios.

    ◊    The program(s) is/are too short to possibly have any error(s).

    ◊    They can be their own quality/testing department.

    ◊    They can/have figure(d) out all the necessary permutations of possible interactions that user(s) would possibly consider entering.

    ◊    There is no budget, nor push for a proper QA department and/or QA procedures.

While the above is a very, very small subset of all the possible reasons that I’ve seen/heard, the reality of it is that these are being given as actual excuses when confronted about what QA processes are in place.  The reality of it is that there are a lot of substandard, unprofessional software developers out there, who despite all the best practices of established community acknowledged developers and software engineers, continue to believe that it isn’t worth their time.

The outlook is sadly very grim from where I stand.  All too often there is considerable resistance from management, and more so from other coders when the topic of writing tests first, something we have been reminded of recently from (most notably) XP and Agile development circles.  Managers don’t want to waste time on a tight deadlined project writing code that will never make it out of the door and to the customer(s), and other coders in so many situations feel it is boring and unnecessary.  

The reality of it all is that those mentalities doom its transgressors to a endless of cycle of bug chasing and failure.  The impression that the end-user/client receives of a given software firm/group/coder is based almost entirely on the quality of their work, but there simply doesn’t seem to be the necessary forethought by those responsible to make the decisions toward quality.  

Quality isn’t simply a department which points out and/all flaws in a product, and quite frankly shouldn’t be taken as such.  Coders, particularly bad ones despise a proper quality arrangement because it points out all of their flaws without ever really providing an equal amount of praise.  Developers as a whole like to hear positive affirmation about their work.  The code produced as, such as an artists, an extension of their being, and thus hearing about problems with their work, they all too often take it personally.  They see it as an attack on their character, and that which makes them who they are.  This is something I would expect of a child, but not a professional.  

It doesn’t have to be this way.  We as a collective group of software developers, engineers and architects are the ones responsible for ensuring that quality of not an afterthought.  We have to make it a priority to build quality into ever facet of not only our software, but every facet of our varied processes.  The size of the application is immaterial.  Whether a simple shell script, or a half-billion lines of code suite, the same level of attention to quality is a pre-requisite for success.  

This includes the planning process as well.  Writing tests before actual code is produced is wonderful, and needs to always occur.  This requires discipline, which many seem to lack and/or overlook as a legitimate need.  It goes further than that though, all the way to the planning stages.  What is the point of having a high quality-focused mindset at the code level, if the project itself is lacking the same on so many other levels. 

We need to demand this mindset from the get go.  It is our responsibility to ensure that the necessary practises are put into place from the beginning, because no one else is going to care, let alone take the initiative.  If you take pride in your work, you need to ensure that it does as it is/was requested, and that requires the right methodology as well as mindset.  Not to imply that our lives depend upon this, but in reality it does.  

It isn’t as if all of this effort isn’t without reward.  We need to teach the next generation of coders to start with quality as their foundation, and it will simply become a ubiquitous piece of the process.  The benefits of this holistic view are many, and among them are:

    ◊    Stable code

    ◊    No surprises

    ◊    Happy clients

    ◊    Time to work on the next big thing (tm)

    ◊    Less stress

    ◊    Future business

    ◊    Peer appreciation

Quality is everyone’s responsibility, and that means it has to start with each individual, no exceptions.  So if you, or others which fit the bill as prescribed above have hangups on this issue, then it is time to think long and hard about it and either hangup your coding chaps, or take that next step and better yourself in your field, kicking your credentials up a notch, by having a quality centric mindset.

In future editions of the codedevl.com journal, I will be taking a more detailed look at each of the phases of the quality aspects of the software life-cycle.

Present Tense English Parser : Part I

As part of an ongoing project during which I have been designing, building and testing in one way or another over the past decade and a half, I have arrived as the parser phase.  Well, I will correct that statement.  I have tinkered with creating parsers before, but thanks to the expressive nature of the Python language, I was finally ready to make a serious attempt at writing an English present tense command based parser.  I'm not going to make a massive post about this, though I am going to post the test results.

Note, all the tests pass.  What a passing result actually means is this; The parsers job as of version 0.5.4 is to break apart the sentence(s) properly into their components via identification of verbs, conjunctions, prepositions, articles, conjunctions, pronouns and punctuation.  

Creating Parser Instance:                                                                                  : Passed

Loading Configuration for Instance:                                                                 : Passed

Testing for version: 0.5.0 

  paint the gold bucket black                                                                             : Passed

  get the big , heavy hammer and kill Bob with it !                                           : Passed

  get hammer and squirrel from Bob and then hammer squirrel into the wall .  : Passed

  get the gold gold                                                                                             : Passed

  kill elf and get gold                                                                                         : Passed

  paint the bucket gold                                                                                       : Passed

  paint the gold bucket black !                                                                           : Passed

  get gold                                                                                                            : Passed

  kill elf , get gold                                                                                              : Passed

  get the large gold brick .                                                                                  : Passed

  paint the bucket gold .                                                                                     : Passed

  get the large , gold brick .                                                                                : Passed

Testing for version: 0.5.1 

  get rock , pliers , hammer and squirrel and hammer squirrel into the wall .     : Passed

Testing for version: 0.5.2 

  kill the trite little elf with my sword , then wipe the blood off of it !                : Passed

  destroy the cantankerous creature before you eat your dessert                        : Passed

  kill the trite little elf with my sword , then wipe the blood off of my sword !  : Passed

  kill the trite little elf with my sword .                                                               : Passed

Testing for version: 0.5.3 

  hammer the hammer into the big hammer                                                       : Passed

  hammer the hammer into the hammer                                                             : Passed

Testing for version: 0.5.4 

  kill the trite little elf with my lavacious sword , then wipe the blood off it !   : Passed

  go to the store and buy a new cellphone                                                        : Passed

  slit Fred's throat and capture the warm , red blood in a cup !                         : Passed

  play with my toys and listen to my music .                                                    : Passed

  play with my toys and listen to music .                                                          : Passed

As can be seen, the variety of possible inputs for the parser vary from simple to complex, from grammatically perfect to questionable fragments.  Being that the purpose of this parse is first and foremost for use in a command environment in which interaction is needed, thus the present tense only requirement.  This is a massive relief on the demands of the parser, but even still, it can be see from the above that the system can differentiate key words which can be used in both noun and adjective forms.  The system also handle post adjective usage.  

The system currently most notably recognises over 9,000 verbs (regular and irregular), 50 prepositions, and a whopping 46,000+ adjectives.  A call for test case phrases is hereby announced.  I am satisfied enough with the stage one parse process that I hereby am moving on to the second parse stage, that is the creation and order of individual statements (as dictated by their prepositions), in preparation for the third and final stage, in which the parser sends the results from stage two to the action engine.  Both those phases will be the subjects of new posts, accordingly.

My Adventures in Software Engineering Consulting

  Starting in August of 2006 I ventured into the world of independent Software Engineering contracting.  Having just completed my previous salaried tenure at a financial firm, and a small side stint for a Fashion magazine as a favour to the owner(s)., I came upon a new offering.  I was hesitant at first because it was a 1099 contract job, a realm I’d never ventured into before in my professional career, and as such the time had apparently come. 

    The position was simple, the company in question was a contractor to a large retail clothier whose name rhymes with ‘Hurlington Boat Factory’ and is located in a town in New Jersey possessing a similar sounding name.  The CEO of the contracting firm requested specifically someone with considerable experience in Python who could build a real-time return transactions processing system which would utilise both XML and Oracle 10g.  While I have a strong dislike (along with many other software engineers) for XML, the reality of getting paid decent money for coding 100% in Python was all that I needed to hear.

    I have to say that I was excited not only because of the prospect of a Python pure coding environment but that I would be working initially and then periodically out of a satellite office in Cherry Hill.  This was exciting because for the first time in my decade plus career, I was finally in a scenario where I was writing code for a company whose primary purpose was producing JAVA software applications for sale.  The office in Cherry Hill consisted primarily of the CEO, a lead developer and a secondary developer.  There were several other individuals who would at times utilise the office including the owners, project managers and other associates of various roles, but ultimately it was an office with other competent coders one of those being quite the master of Cold Fusion technologies.  

    After my first few weeks on site in the Cherry Hill office it was time to start working on-site at Hurlington’s headquarters along with the only other consultant coding in Python.  This was the beginning of a very enjoyable period in the contract due to the excitement of the project and the joy and experience of worthing with another Pythonista.   

    The details of the project are much like any other project you might encounter when modelling a new project off of a customer version produced for a specific client.  It got ugly for a while due to the multiple aspects of the project including client systems which needed to be integrated for proper functioning along with an existing array of registers distributed nationwide and a database number records approach a half a billion entries.  The normal in fighting and finger pointing existed but it was all worked out in the end with a finished product delivered and a contract satisfied 100 percent. 

    The only issues encountered which left a bad taste in my mouth were those pertaining to a clueless project manager who regardless of his claim of years of experience was apparently lacking considerably outside of his realm of Oracle, which caused unnecessary attitude due to his ignorance of coding and APIs.  There were other issues dealing primarily with suits, but I doubt that this specific issues varies much anywhere.  Suits and Engineers rarely if ever mix, let alone get along except on a faux-cordial level.   The ultimate end of the contract was due to the project portion I was responsible for coming to fruition, even though I was already working on leaving regardless.   It wouldn’t have mattered much anyway given that the main contracting company is located up north in the centre of New England and are in the process of relocating everyone to that locale, an act which I am unwilling to do.

    I do like certain aspects of software consulting, but given that there is a considerable amount of extra taxes which must be held aside, some unholy hours which must be adhered to for the purposes of meeting quite strict time lines, and the reality of being a second class citizen in the work environment (or 3rd class in my case as a sub-contractor).  It all comes to an end in two days and I go back to a normal, preferred work environment in five days when I start up in a salaried position again working for a web host writing a multitude of applications and revisions of existing software, in a nice relaxed geek environment, also in New Jersey, but this time around, I’m excited to not be a consultant.   At least for the time being.