Why Defined Coding/Operational Standards Matter in Modern Software Engineering Environments

After a brief stint in the hellish 1990's world of programming in Perl for a half-billion dollar company for a few months before the realisation that it was indeed a living code base designed and maintained (I used the word loosely) as if it were still two decades prior, I accepted a new client in the position of Lead Software Engineer & Architect at a local company in Montgomery County, PA, USA.

Much akin to what I experienced at the aforementioned company, I found myself in a microcosmic variant of my former employer save for the fact that the system I am to replace is written in ASP Classic on an older IIS windows centric system.

These situations are not at all uncommon in the real-world, and hopefully they are diminishing at an increasing rate.  This is not me condoning the carte blanche manner of always re-writing an existing system from scratch whether than includes changing the existing infrastructure (for better or worse).  Each instance/environment must be assessed individually along with company goals, available budgeting and forecast to ascertain whether or not such an approach is viable.

Bringing this back to the core reason for this post is that a well defined coding standard didn't exist for either environment, not to mention a revolving door of people assigned and/or employed which had a hand in touch either system.  Both environments differed in that one was a fairly modern agile approach with daily scrums, open communal work spaces and solid department leads for projects and that the other was very much waterfall design based, separated (officed) individuals with little communication and a lack of leadership in terms of working environment understanding (though a solid understanding of the business logic and customer relations carried this disconnect for without it, all would have faltered).   Neither environment possessed nor utilized any semblance of coding standards.

What kind of standards are we talking about here.  The whole gamut is my best response.  How are function names derived, how are attributes accessed, how are constants, classes, variables, methods, etc. named (verbs for methods, ambiguous/nonsensically, etc), typed (cAmElCaSe, alllowercasewithoutbreaks, properly_divided_with_underscores, ALLUPERCASE, MixTUREoFMeans, wthtvwls, etc.).  No set rules for braces or brackets (K&R vs same line), indentations (2, 4, tabs, spaces?).  Lengths of lines, length of method calls before being broken down.  Use of native data constructs.  Rampant string concatenation and massive use of type casting instead of a well kept control of expected content.  Complete lack of logging and little if any use of exception handling (where available).  Poor hiring practices which allowed for a single point of decision to be made either based on a inconsequential question about the language du jour at a given company or because the hiring manager lacks any breadth regarding software engineering both in practice and theory and does not do group interviews with potentials as a precautionary measure.  Lack of enforcement (or implementation at all) of code versioning systems/controls, proper QA practices, etc.

All of those situations were (and still are) prevalent in the existing systems, as well as many more which I will not get into now (solely because there isn't enough brain bleach to forget that they still exist.)   What I can do is point out that because of the lack of all of the above, both companies (while still in existence):

- Find themselves over the years running into longer lead times for changes which would otherwise be quick and simple.
- Experience bug fixes which require lead times of weeks due to lack of documentation, version control, communication amongst peers and of course proper reviews up front before production pushes.  - Struggle to keep current due to lack of faith in the infrastructure of existing systems that any new code would cause massive breakage and production down time.

This is never the way to run a company and after roughly two decades in the field professionally, it is disheartening to have to endure.  There is a light at the end of the tunnel.  The former company mentioned has a few good remaining people that provide a glimmer of hope for change in the right direction if they can convince the powers that be (ego and all) through all the crimson coloured tape that  these changes are necessary which could lead to the arduous process of modernizing.  The latter company referenced has already made that investment both with full ownership backing down the entire hierarchy of the org chart, and in that case, influenced heavily by yours truly.  I see much greater things for that latter company because it has done the realistic self assessment and see that much is lacking, ultimately choosing to do something aggressive to resolve the situation with a great outlook towards future business.

Hopefully we can see more and more existing companies learn from these examples before they find their up and coming competitors who embrace a method to their madness soaring by and taking their bread and butter in the process.

Rediscovering an old friend, Perl.

It has been over half a decade since I was paid to code in Perl, having since moved on to Python amongst other languages.  I use the past tense because this is all changing once again.  Due to circumstances that are almost entirely personal-life related, I find myself moving on from Yorn after the past fifteen months of doing great things with wonderful, intelligent and foward-thinking people.

After months over two months of screenings, phone interviews and an on-site meeting, I have accepted an offer as Production Lead Engineer at Fanatics, LLC at their Conshohocken, PA facility.  I'm going from working in a Python/Django environment for the past half decade plus into a more systems/internal middle-tier/core environment based on an old friend, Perl.   And despite some of the issues which led to this change, I'm wholeheartedly looking forward to it, and here's why.

As we grown our personal knowledge in a specific discipline, in this case programming languages, our perspectives and approaches mature.  When we step away from our comfort zone into some new experience (or language in this case) as I did professional when I moved on to Python as a requirement of Retail Expert Inc, CTO (at the time) Andrew Sernekos, I shifted mentally and learned the way of the Pythonista.  As the years progressed it became the 'one true way' of thinking, which was fine given that my language dictates were in-line with my choices and professional tasks.  I had put aside my TMTOWTDI (there's more than one way to do it) mantra from my Perl Monger/Perl Monkish days of yore.

Refreshing my skills back into the primary language through which more than half of my professional career was built, Perl, has opened my eyes in a way not previously possible.  The memories I had of Perl with its heavily flexible manner/varieties of expression had originally left a negative taste in my mouth but now having disciplined myself cutting through Python, Groovy, Scala and others has proven to expand my view of what Perl truly holds in terms of power.  With great power comes great responsibility and now with the modules from CPAN such as Moose and its derivatives (as an example), the future is looking quite fascinating again for me with a language that has been around now for 35 years.

Here's to the joys of the future and best wishes for the incredible team at Yorn as a sail on to my next contractual adventure.

Theorising vs. Doing

Over the past many years I've found myself slowly moving (in my personal projects) from the realm of actually implementing code about which I've been curious to spending my overwhelming majority of my free time to theorising about various concepts and/or interests, primarily in the field of human language parsing, simulation and simplification of generic real-world object modelling.  Several of these have been mentioned and explored in previous posts over the last 7 years.  

I believe a great reason for this change in focus is partially age, and primarily due to the extent that my days are focused on building the next great startup application.  We recently (as of the New Year) pushed the past eleven months worth of work into existence as our new primary (Python/Django) product replacing the previous multiple iterations of our legacy (Java) product.  The focus and scope of the project limited my ability to go off on tangents wherein I could code my personal projects freely.  This was by no legal hand binding, it was simply a matter of wanting to focus my writing to building our product to the level it needs to be, to the high standards we require (and rightfully so).  

This leave one (me, namely) with little actual time to thunk down in front of my primary workstation to commit theory to codebase, hence I've found myself working it out on handheld whiteboards, sketchpads,  napkins, chalk on the ground whilst playing with my daughter or even simply working these theories out in my head.  While I would like in many cases to put my thoughts into actual runnable logic, I've found that the exercise of stepping through theoretical code in my mind has kept me sharper regarding my thought processes.  True, I cannot share this as easily with friends afar, but given that many of my personal friends are also senior level engineers, discussing (when we do) casually my theories and ideas, they get the gist quickly.  

The point I'm trying to get across here is that while today is perceived as one wherein a bunch of 20 year olds either in or fresh-out-of college spend an obscene amount of hours in incubators or at startups killing themselves churning out ridiculous amounts of code, there is most definitely no slow down in the thinking processes of the more senior of us out there.  We just don't find a necessity it raw churning out of code as it is the concepts that drive the field and aid in future innovation.  Let those who wish to "do" continue to run their path, but lets not overlook the value of those who've moved their focus (whether by situation or by intention) to the more abstract realm of theory as the two are inextricably bound.  We need ensure that these two distinct groups of people are in contact with one another because it will ultimately lead to the newer advances, bettering our field for all involved.

New environments (mentally) as it were... (a.k.a. something old, something new)

Since my last update made a little over a month and a half ago, much has changed.  I've accepted a position as Lead Software Engineer for Yorn, LLC located in Conshohocken, PA.  I haven't abandoned my prior position with the publishing house in NYC, merely time-shifted those hours elsewhere.  Taking this new venture (on top of my previous work, plus the other tech-startup of which I'm a co-founder, Ecquire has proven to be a necessity for my sanity.

Working primarily for the past many years as the sole Software Engineer for a half dozen magazines in a Django/Python/Javascript/CSS/HTML/FreeBSD Unix environment whilst also wearing the SysAdmin hat has been...  draining.  This wasn't due to having nothing to do, this was more to do with having very little room for innovation as the end product wasn't the new SaaS website or Outlook plugin, it was simply repetition with the only real variances being that as dictated by wet-behind the ears designers and graphic artists whose concerns about a given site was more about look than UX.  Don't get me wrong, some of the designs have been gorgeous, but not without the pendantic whining common with individuals who only knowing Adobe Photoshop aren't happy when you explain to them that there is a different methodology at work in producing like-layouts im a web framework.

Either way, back on topic.  I feel for the first time in quite a while that not only am I solving real problems that pertain to a large scale of individuals and companies (as opposed to readership by fashion and/or tattooing enthusiasts) but I am finally in a position in which I have a cohort in crime, a partner, a colleague. Best of all, he's brilliant.  A proper individual for the field, something when I've been trying to find for ages.  A PhD in computer science and a quarter century in the field along with all of the traditionally odd hobbies and sense of humour found more commonly in those with very intellectual professions.  Even though we've only met face to face on three occasions, we've shared many phone calls and Skype sessions whilst working out our designs and product development and it has simply been a breath of fresh air for me.  I have someone off of whom I can bounce complex ideas knowing full well that I'm understood, as well as presenting me the opportunity to expand my own learning horizons.

Excitement stirs ones involvement in a given project and considering that this is what has (and is) happening to me, it is paying off in not only a clearer mind and what I believe are more robust ideas emminating from my thoughts but it has reduced my stress allowing me to be more productive all around whether in the publishing work, my own startup and more importantly in my family life.  It all comes back to ensuring that whatever one does, they need to sometimes step of a comfort zone in pursuit of that which provides incentive and drive in ones given interests and/or profession(s).  I was too conservative earlier on in my professional career and I believe I would've benefited greatly had I stepped out of my cocoon years earlier rather than (as most introverted engineering types do) not think highly enough of myself to be worth more than an abysmal income and working environments to which I was subjected for over seven years.

Times have changed and the future is looking brighter and brighter.  Ecquire & Yorn have paved the way to a better future.  I stepped past my initial nervous/worried leanings and jumped into the world of proper tech startups finding that the grass truly has shown to indeed be greener on the other side.  Now, onto another exciting and challenging day of practising my craft.

2011 and my prospects...

    Having recently entered the new year, I find it fitting to share some recent happenings and thoughts on how they might apply to not only my situation but those of others with a penchant for software engineering and/or administrative careers.

    The past year has been one of making connections.  I have met some great people including several combinations of Angel Investors, Venture Capitalists, Software Engineers and Computer Scientists.   My experiences with our startup, Ecquire (http://www.ecquire.com) has proven quite enlightening as well as fun.  In fact it was through my co-founders I was introduced to so many others in the field, such as Rick Rasansky and Trip Denton of Yorn (yorn.com), and Gabriel Weinberg of the amazing search engine DuckDuckGo (duckduckgo.com) which protects our privacy.

    What I have come to realise is that while I have been working with software behind the scenes of a considerable number of websites on their various frameworks, the technology doesn't challenge me and has proven to be quite repetitive.  There are still some challenges, but there is far more aesthetic issues than technical ingenuity involved.   The issue of technical seclusion is also an issue at hand.  With my primary client for whom I have a special relationship due to past employment in the earlier half of the previous decade, there is a loneliness on two levels.  

    The first being that I am not only the only Software Engineer, but the only technical person period.  I wear many hats including CTO and Systems admin.  I handle all technical contracts for servers and what not, at least related to any only presence.  I'm more than capable of handling these roles though it can be rather isolated and not because I'm physically in a different location.

    Which leads me to the second reason, that being one of no technical peers with whom I can discuss, debate and/or share views on projects.  The owner of the various entities is more understand on technical issues that many with whom I've dealt over the years but there are still limitations of understanding.  Again, I do highly commend him for intelligence, trust and understanding though there are limits of which no one other than a peer would quite grasp simply because it is something that needs to be experienced for commiserating to be legitimate. 

    As a means of striving for more in not only personal life, but my professional one as the combination of both affect my happiness and general stress levels, I've been partaking a re-design/porting job with a technical group which is more up my alley.  I am still delivering the service and quality to which I have committed myself with my primary clientele, let that be stated for the record.  It is just that I need this to keep my mind from atrophying.  I need to be challenged in my field, not to mention be given the opportunity to innovate without having to wear every hat and focus so much on outward aesthetics.  I like  beautiful design and beautiful code, much so less visuals as that is truly the realm of designers. 

    Where I'm coming to for the sake of others is the following:  Push to keep yourself challenged and happy in your field.  For a long time we allow ourselves (especially as we have more people depending upon us in our personal lives such as our families) to become complacent with the idea of security and daily consistencies in our work.  The problem with this is that we start to stress over the rut in which we find ourselves.  We internalise the urge to snap at others when for the millionth time (exaggeration of course) we painfully banal question or task is asked of us.  Though inside, we die just a little bit more each time, in spirit at least.

    The solution (or at least one solution) is to not let fear or complacency hold a firm grasp on us.  It is one thing to allow for short term bursts of such ideals for the sake of a greater good but to allow it to trap us long term in what is almost definitely a downward spiral is unbecoming of our abilities and intelligences both as individuals and collectively as the technical backbone of our industries respectively. Push to find new challenges and if they are drying up as a lake in a newfound desert, look elsewhere over the horizon as our abilities only go so far when our outlook and prospects of change seemingly whither into nothingness which ultimately leave us a shell of our former glorious selves. 

    I'm not taking this new year lying down.  I'm feeling reinvigorated with new purpose, prospects and challenges not only for the betterment of my familial situation but for my peace of mind now and into the foreseeable future.  I wish the same to all of you whom finding yourself in a similar situation might find my words useful and/or inspirational.

Romanticisation, Research & Reality

As I've now hit that point in my mid-late 30's, I've had some internal conflicts recently see resolution.  This won't be one my traditionally long winded/verbose entries, though it won't be short enough for a micro-blog post either but here it goes...

Over roughly the past two decades I've been torn by issue of not attending university, especially one such as MIT, Stanford, Virginia Tech or UC Berkeley in pursuit of a Masters/PhD in computer science.  I feel that I missed out on the social aspect (aside from the excellent environment for furthering myself in a group setting in my field of interest/passion/expertise.   This is simply a recurring romanticised view I have over my regret.  I only envision the good parts, not the tedious drawn out study periods, the painfully boring pre-requisite programs/classes and lecture in mundane subjects solely to satisfy the higher education machine.  I am an autodidact and a fairly effective one at that.  University would have most likely quickly become the bane of my existence.  I experimented with higher education and within the first semester found the pace horribly slow and in the case of required classes, a horrible waste of my time and money (I paid the tuition with my own cold hard cash).   I opted to leave before wasting any additional money and simply continue to do what I had always done, educate myself for a variety or sources, as well as getting my hands dirty in my field of interest (amongst others for well-roundedness).

This brings me to one of the reasons which fed the romanticised ideal as prescribed in the previous paragraph(s); my research in the field of immersive virtual environments.  I have spent a considerable amount of time since my childhood in the areas of researching virtual environment simulations (e.g. simulations of real-world objects and scenarios).  This could entail human interaction with every day objects to be used for gaming or habit studies or to simulate atoms at the molecular level or even the macroscopic planets, interstellar bodies and galaxies in our and other places of the universe.  This is the kind of research which would have either lent itself towards achievement of my PhD or conversely, the focus of my post-doctoral research under a grant.

Finally, the reality (hint: it isn't all that bad).  Working in an academic environment in professorial/research dual/split roles would've been a wonderful way to go, though it wouldn't allow me the flexibility I currently have in terms of working from my home whilst raising a toddler, and being home for my elementary school aged child as well.  Do I get funding to pay for my personal pursuit/continuance researching simulations and their many facets?  No.  Do I have a surplus of time to allow me to pursue said interests after my family and work related endeavours are satisfied?  Not much at this point in time.  Do I feel that the overall balance of what could have been versus what truly is and what still may come my way is reasonable, fair and not worth feelings of regret?  Yes.  I'm rather happy with where I am as a Software Engineer for the past 15+ years and while I'd still love grant money to further fund my research, I can't say that I don't  enjoy some of the daily challenges of my already existing present-time workload.

Just because we might find ourselves in a situation that isn't exactly as we'd initially thought we would want doesn't mean that there aren't equivalent outcomes that still satisfy our initial hopes and aspirations.  I suggest that those of you who have dealt with the aforementioned conundrum, take a good long look at how you might still achieve the equivalent that fits best into your existing life plan and stop worrying so much about such exactness in realisation of ones dreams, otherwise they might never materialise in any recognisable form.

The Usefulness & Addictive Nature of Multiple Displays

It seems to be a divisive issue when dealing with developers, engineers and programmers alike... The usefulness or annoyance of multiple displays.  Recently having upgraded to my fifth display, I felt it was about time that I weigh in on the issue/share my opinions.

Five displays : One 1920x1080, Two 1920x1200, One 1280x1024 and One 480x800 :: 8.4 Mega Pixels in All

Back in 2001 at an interview with Health Market Science in Conshohocken, PA (now in King of Prussia, PA I believe), I had witnessed a key benefit of new up and coming startups for their developers...  multiple displays for each workstation/desktop.  Flash forward a few more years and one Mac Book Pro later and I find that ever company at which I would work, I would attach a secondary display so as to provide more desktop real estate.  Whether this was to provide a shell or two in which to run emacs while accessing a db frontend or browser preview window, it proved beneficial in terms of productivity with less switching between screens, windows and what not.

Ever since my first exposure to such setups (which were astronomical in price to setup back in the 90's and the beginning of the newest millennium), I was rather enamoured by them.  Now in 2010 I find myself working not on laptops, but full blown 64 bit Unix workstations.  A major benefit of such setups is the ability to run a considerable number of graphic cards and their associated displays.  Some wonder what could I possibly have running that dictates so many displays.

I personally like to keep a web browser open at all times so using the picture above, I will explain as best as possible.  So, as I was stating in the previous line, the left most display is running a full height copy of Google Chrome, flanked by three separate terminal windows running a mix of local and remote shells on  various servers.  The middle large display is running my current favourite IDE for python development in fullscreen (in this case, NetBeans 6.8).  The right hand most screen is used for additional remote shells used more so for large rsync'ing and process monitoring.  The upper left screen (a great displaylink/USB powered 480x800 (or 800x480) from Mimo) is used for Skype & Adium (and both are pinned to all spaces/virtual desktops due to content), and the upper rightmost monitor is a free for all to display whatever is needed in addition to all of the aforementioned items (pdf viewing, techtalk viewing, the occasional movie, db schemas or gui, etc.).

The biggest issue I currently have with the setup is physical layout.  A table roughly 144 cm (60 in) wide can only handle three widescreen displays (between 23" & 24" diagonally) when in a concave configuration.  Having to position the additional displays above the lower displays does cause the occasional confusion regarding pointer location.  A better layout would be an array of identical displays, preferably with VESA mounts in a 5 wide portrait layout or a two rows of three in landscape layout.

I highly recommend that others who haven't had the opportunity to work in multiple display setups to try it out as soon as possible.  Those simple keyboard shortcuts used to swap from window to window and/or virtual desktop to other virtual desktop do take up time and can interrupt one's flow.  Is it really worth it when displays are so cheap these days?

Caveat emptor:  The use of multiple displays causes varying ranges of discomfort when penned/cornered/shackled into a single screen machine, not to mention there is desire to acquire more screens after getting acclimated to the first addition, second addition, etc.

Till next time...