The seemingly separate world of tiered development enclaves.

I realise the subject of this post is a bit of a mouthful but it was the best I could do to summarise what I'm thinking.  In the world of software development there are often a series of cliques based on languages and/or platforms.   In other words, we have our Java/C#/Python/Ruby/Erlang/Scala/etc. running on Solaris/Linux/FreeBSD/Windows platforms.   There are tight knit groups fervently gathered around almost any combination of those two collections.

The impetus for this rambling is that there are expectations that each group places upon everyone regardless of group.  For the Java/C# crowd, as an example, there is an expectation of certain protocols and mannerisms of doing things such as the use of SOAP and XML for webservices or the Python/Ruby/Node.js equivalent crowd touting heavy use of JSON and ReST based systems.  

What amazes me is that the more closed in a group, the greater the expectation that the world does everything the same way seems to grow, not shrink.  Note that this doesn't imply everyone, just a surprisingly larger number than anticipated (by a long shot).  I don't mean to imply this is across the board with no exceptions.  People used to working with web services in Java utilizing SOAP and XML seemingly care little (and know less of) the more current alternatives such as JSON and the various frameworks and standards.  This is a problem because it leads to an adversarial misunderstanding which I think holds back collaborative situations as well as opportunities to expose individuals into a more cross environment manner of function.  

I know that with my near-two decades of professional experience that I have worked in a core of languages (Perl, PHP, Python, Javascript, Ruby, ARexx, Pascal, etc.) with a series of acronym based technologies and formats (JSON, XML, PostScript, YAML, etc.) but that there are people out there flabbergasted to find that I've never had to work with CORBA, SOAP, XSLT, COM or even something as innocuous as certain frameworks whether Cake, Struts, Catalyst, .Net or Seaside.  

The reaction is often one of shock and confusion because for so many, the entire world revolves around a solid set of standards, practices, protocols and languages which without, so many would feel like a fish out of water.  In reality there are only so many ways to write an 'If loop', or an iteration over objects in an array/list/hash/tuple/set, etc.  There are only so many ways to re-implement MVC frameworks or 3rd party libraries.  

I often make it a practice to keep current with the langauges, frameworks and protcols of my not-so-close (codebase wise) friends so my familiarity is there, and has actually benefitted me during technical interviews over the years.  This ties into what I'm saying, I will come back off of this slight detour.  I was at an interview for a perl position at a large development company in the King of Prussia, PA area back in the early 2000's and it involved a 6 hour interview process on-site.  I will spare most of those hours but there were two 45-60 minute interviews that stood out.  The first was on 'C' followed by 'Java', neither of which I've coded professionally (or personally for any non-learning period of time).  Keeping current did help me muster my way through the concepts and what not, albeit barely.

This is where I bring it back to my original point for this post.  A couple of years later down the road during a technical interview for a job in a language/environment which does not have the concept of 'interfaces' (as in the C#/Java sense (to name a couple)), I was asked such a question.  I first pointed out that interfaces don't exist in the target environment as it does in C#/Java, and then proceeded to give a crude explanation on how those are used in Java.  The sound on the other end of the interview table was subtle but noticeable; One of confusion as to how someone doesn't use such constructs (or the fact that not all languages/platforms uniformly implement the same exact concepts).

It really comes down to education, in both directions (and I'm not implying academia to be clear).  Engineers should be able to keep their familiarity with other languages/platforms current enough to be able to understand those difference between both their local toolsets and those of everyone else (to a reasonable degree).  Just because I don't code in Scala professionally doesn't mean that I shouldn't read articles about Scala's 'traits', or that I should ignore a different write-up on Groovy on Grails despite not using Groovy or Grails.

This also means that those engineers of the Microsoft stack should at least familiarise themselves with what Django is, a bit of Rails, maybe some PHP and Perl, a little bit of Lua and Lisp/Clojure/Scheme, etc.  In a former situation I ended up taking the development lead in a Microsoft heavy envioronment writing Classic ASP for the first (and preferably last) time, so I've practiced that which I preach.  Knowing some of the pieces that differentiate the languages also give insight into work and processes that may be missing from our own stack(s).  Our corpus of knowledge collectively shouldn't have such extreme gaps from discipline to discipline because it as a whole divides us from possibly creating the next big thing as well as (maybe more importantly so) fulfilling our individual potentials as developers/engineers/creators.

So take this as an opportunity to visit that neighboring x/y/z technology users group, maybe even a smaller convention/conference to get uncomfortable yet ultimately expand one's own horizon and knowledge-base.  Only good things can come of this.  Continuous improvement doesn't solely refer to processes and applications but people as well.

They Say That Time Will Tell.. Ruby Revisited - Part IV

I've written on multiple occasions regarding my flirtations with Ruby (the language) and my ultimate feelings of it falling short of the mark.  I first experimented with Matz's creation in 2003, some 8 years after his initial release.  My first foray was not for professional purposes, but exploratory as a means to best express what would eventually become the SimulaE project which I ultimately crafted in Python.   I did end up writing a machine learning/route optimisation experiment mimicking hospital utilized medicine delivery robots I'd seen at Abington Memorial Hospital in the prior months.  

My conclusions were that Ruby was still too TMTOWTDI than I'd liked.  It all smelled heavily of Perl, which after using for almost ten years (at that point) was less than desirable.  This was long before Rails existed, and it would be another five years before I took another serious look at the language (post Rail's introduction and the famous screencasts which accompanied it).  

Yet here I am five years further down the road from my last foray with the language named fondly after a gemstone.  At this point, I've been coding professionally for over eighteen years, and in general for well over three decades.  I've grown up quite a bit and realised that it was my own issues and pig headedness along with some errant expectations which led to me ignoring Ruby and embracing Python at being closer to the hypothetical "one-true-language."

Years of writing in the BDFL's (Guido van Rossum) creation stemming from the ABC language has done wonders to clarify my deeper understanding of engineering principles, clean code design as well as inconsistencies and even MVC frameworks (Django from v.96-v1.4+).  I believe that it was actually through my understanding of Python interspersed with larger monolithic projects in Perl that drove to my moment of clarity regarding Ruby.  

Don't get me wrong, it wasn't solely the above which led to this epiphany though.  What really finalised it for me was the wonderful Ruby Rogues, the Ruby Freelancer's Podcast and various other podcasts involving Charles Max Wood.  The sheer vastness, diversity and information, knowledge and experience sharing within the Ruby community caught me... hook, line and sinker.   Note: This is a different community (from my own observations) than the Ruby community of Christmas Past as 'eloquently' espoused by the always-blatantly-honest-with-words Zed A. Shaw.

I won't go into the details about what it is in general that made me finally get Ruby this time around but I will say that the recent improvements in the language as a whole and the more natural flow of constructs and method chaining resonate with me as a engineer and polyglot.  The efficiency of :symbols, the consistent smalltalk based mannerisms regarding method invocation and lest we forget the flexibility of code blocks.  I simply leave the reader with this.  Check out the links I've provided.  Listen to the podcasts, try some of the exercises and give an honest assessment of what Ruby has to offer.  I'm really glad I did and now wholeheartedly look forward attending the next Ruby convention which avails itself to my schedule. 

Invoking the inner Howard Roark, for the Sake of one's sanity.

I was listening to episode 059 of the Freelancer's Podcast (formerly the Ruby Freelancer's Podcast) during which the discussion of handling burn-out arose.  That issue in itself hasn't been a matter for me for many years as I've learned how to balance work/personal/familial/etc time quite a few eons back.  It has even been referenced from other associates and one fairly recently (see: Tal S. Raviv's "Customer's Over Code" blog from the founder of Ecquire.com at http://talsraviv.com/2012/10/21/burning-out-startup-founder/), but it did get me thinking.

What really stood out about this podcast was how on-the-ball the hosts seemed to be regarding controlling expectations and ensuring a balance in one's life vs. work.  You are not your job, though that's not to say you cannot be passionate about it, or that it doesn't play a role in who you are as that would be a blatant lie.  In our profession (I cannot speak to others' professions) I have seen time and time again the level of life breathed into both companies and projects alike with all the eagerness to continue efforts with a constantly renewed sense of vim and vigor.   Again, this is not a problem.  The problem is when factors take all that is good in these delicate balanced recipes and throw the odd spanner in them, messing up the works.

For sake of argument, I'm going to give two examples of environments non-conducive to keeping that balance of personal/work and ultimately happiness and what it takes on the personal level of the engineer/developer/insert-professional-role-here to make it work.

Firstly we start with the bad: company F.   This company is a large established corporation with a half-billion dollar annual revenue and a considerable amount invested in technological infrastructure.   The core problem?  At it's core a system that was written by a non-developer some two decades ago  that served its purpose in smaller scale but when retrofitted to grow to the demands of a huge system against its design required a staff of fire-fighters.  Sometimes this happens but it usually involves a means of correcting the core issues not just throwing more fire-retardent chemical upon its base.

Company F is further harmed by sentimentality and assumed-correctness by the original developer, now a executive member, which has caused questioning of said codebase to be verboten.  Compound this furthermore by developers, many on the team who are quite capable and talented not knowing how to say no.  This lead to an expectation of 60-80 hours a week being the norm which is contrary to all of the known credible studies which show effectiveness diminishes substantially and error-creep rising sharply after a certain amount of hours in a given day be applied directly to development functions.  Self respecting developers who are worth their résumés' contents don't fear for their jobs as if it is the only one available to them.  They don't stay in situations where this is such an unwillingness to make the necessary changes to a core system is present due to one individual's sense of emotional connection to what they feel embodies their greatness/mental superiority from days long past.  Good developers and engineers recognize the situation for what it is and leave for greener pastures or they self-immolate trying to play role of code martyr/firefighter.

Secondly we come to a whole different beast: company E.  This is a much smaller company but of the same age as Company F albeit with an annual revenue at a dwarfed 2.5% of the former.   Company E was a brick and mortar business which traded hands via acquisition through the years and despite having a technology infrastructure, it lacks any actual capability in its development department's management.  A tar-pit that many older company's have falling into has been similar to this:  Making a conservative move into development in-house but re-positioning unqualified individuals into decision making roles and leaving them there for so long that they have no only garnered an unwarranted reputation for having skills they never acquired, but they start to believe they are on-the-ball and a player in the field.

This is one of the most dangerous scenarios one usually encounters (in most any field).  Someone who knows just enough of the bare minimum to be horribly dangerous because they can sound like they know what they're doing to the higher levels of management that are clueless on matters such as this.  This leads to some very large sins in the industry, over promising product and providing timelines which have no basis in reality.  These are company relationship killers.  This is a scenario where the individual making the decisions think they know when in reality they are the least knowledgable,  yet their wield a title (and that air of knowledge wrongly attained over time via entropy of concern by those around not directly in the field).

The solution that mentality leads to is similar to company F's perpetual firefighting, but worse on several points, primarily being a smaller player in the field therefore being more fragile if deadlines are missed and secondly, though equally as important is an expectation of the all-hands-on-deck (lack of) work ethic also in perpetuity.  This leads to low morale, a complete lack of trust and a decline in productivity overall.  Software developers/engineers are smart individuals and some of the largest technical companies have realized this so far to the point of letting the animals run the zoo (removing managers) with generally greater success.

Where am I going with all of this?  Expectations work both ways, and they must be managed.  This is where the Howard Roark reference comes in (for those not familiar with "The Fountainhead" by Ayn Rand from their high school or college days).  In this profession there is oft times a dichotomy between managers and engineers who are ultimately trying to accomplish the same end goal.  All too often it is at the expense of the creator/engineer/developer by those who gain their positions of prominence through abuse of that relationship and a one-sided over-bearing controlling of expectations.

How did I get here from where I started this post?  In the Freelancers podcast, it was stated something along the lines of "I inform my team members/company with whom I'm contracting that I have obligations and will be leaving at 6pm sharp."  I too have these obligations and have made it very clear over the years (especially as I grew in my abilities and experiences) that while I have a love of building great product, and that I am paid to make money for those to whom I've contracted, that I also have other responsibilities that I hold of equal or greater value and will not allow those to be intruded upon.  I  am also not saying that there isn't any flexibility, and I say this as someone who worked for three startups simultaneously whilst raising a new born, I can personally attest to having slept consistently under four hours nightly for over a year's time.  I put in the necessary time, but I have a heavy say & input on when that additional time is implemented due to my other highly valued responsibilities.

On a related note, every time I hear a junior developer offering their own personal time (on a regular basis) to finish a project or start a new one I get frustrated.  They're being played for a fool and those doing the playing (the managers, etc.) will reap rewards for the other's 'dedication' and uncompensated offering of time and energy.   A pat on the back or a $10 gift card for what in billable hours would total something closer to $750.00 is an insult.  Atta-boys are nice, but not at the expensive of dignity lost due to failure to accompany said praises through proper reimbursement financially.

We must hold steadfast to principles that guide us for our own health's sake, our sanity and our livelihoods respectively.  This requires being willing to turn down the great sounding contract, or resigning from a role wherein that kind of abuse of working relationship has been introduced.  When we're younger with less of a portfolio, knowledge base and considerably more naïve, we see this as a rather scary suggestion.  It is only once time has passed, that many of us realize that we're the ones with the power as we have the skill set that solves the problems, produces the great product and ultimately wins accolade's for & from our client(s).  We need to continue to demand the compensation for the fruits of our labours and not let that compensation be seen as a benevolent gift, guilting us into indenturing ourselves to do further uncompensated works out of some emotional bondage.

We need to see ourselves as the professionals we are.  This does not come without us as individuals demanding the respect for our roles and capabilities that we indubitably possess.  We can do great things and overcome seemingly insurmountable tasks/projects but for the sake of our respective individual health, sanity and our art, we must do it on equal footing, not someone else's unrealistic dictated terms.  We are all business people forming contracts with one another, it is simply up to us to ensure that we're not getting the short end of the stick and even more so important to ensure that the individuals on the other end of those contracts know this as well so as to end this abusive cycle of unrealistic expectation and abusive of professional relationships.

Why Defined Coding/Operational Standards Matter in Modern Software Engineering Environments

After a brief stint in the hellish 1990's world of programming in Perl for a half-billion dollar company for a few months before the realisation that it was indeed a living code base designed and maintained (I used the word loosely) as if it were still two decades prior, I accepted a new client in the position of Lead Software Engineer & Architect at a local company in Montgomery County, PA, USA.

Much akin to what I experienced at the aforementioned company, I found myself in a microcosmic variant of my former employer save for the fact that the system I am to replace is written in ASP Classic on an older IIS windows centric system.

These situations are not at all uncommon in the real-world, and hopefully they are diminishing at an increasing rate.  This is not me condoning the carte blanche manner of always re-writing an existing system from scratch whether than includes changing the existing infrastructure (for better or worse).  Each instance/environment must be assessed individually along with company goals, available budgeting and forecast to ascertain whether or not such an approach is viable.

Bringing this back to the core reason for this post is that a well defined coding standard didn't exist for either environment, not to mention a revolving door of people assigned and/or employed which had a hand in touch either system.  Both environments differed in that one was a fairly modern agile approach with daily scrums, open communal work spaces and solid department leads for projects and that the other was very much waterfall design based, separated (officed) individuals with little communication and a lack of leadership in terms of working environment understanding (though a solid understanding of the business logic and customer relations carried this disconnect for without it, all would have faltered).   Neither environment possessed nor utilized any semblance of coding standards.

What kind of standards are we talking about here.  The whole gamut is my best response.  How are function names derived, how are attributes accessed, how are constants, classes, variables, methods, etc. named (verbs for methods, ambiguous/nonsensically, etc), typed (cAmElCaSe, alllowercasewithoutbreaks, properly_divided_with_underscores, ALLUPERCASE, MixTUREoFMeans, wthtvwls, etc.).  No set rules for braces or brackets (K&R vs same line), indentations (2, 4, tabs, spaces?).  Lengths of lines, length of method calls before being broken down.  Use of native data constructs.  Rampant string concatenation and massive use of type casting instead of a well kept control of expected content.  Complete lack of logging and little if any use of exception handling (where available).  Poor hiring practices which allowed for a single point of decision to be made either based on a inconsequential question about the language du jour at a given company or because the hiring manager lacks any breadth regarding software engineering both in practice and theory and does not do group interviews with potentials as a precautionary measure.  Lack of enforcement (or implementation at all) of code versioning systems/controls, proper QA practices, etc.

All of those situations were (and still are) prevalent in the existing systems, as well as many more which I will not get into now (solely because there isn't enough brain bleach to forget that they still exist.)   What I can do is point out that because of the lack of all of the above, both companies (while still in existence):

- Find themselves over the years running into longer lead times for changes which would otherwise be quick and simple.
- Experience bug fixes which require lead times of weeks due to lack of documentation, version control, communication amongst peers and of course proper reviews up front before production pushes.  - Struggle to keep current due to lack of faith in the infrastructure of existing systems that any new code would cause massive breakage and production down time.

This is never the way to run a company and after roughly two decades in the field professionally, it is disheartening to have to endure.  There is a light at the end of the tunnel.  The former company mentioned has a few good remaining people that provide a glimmer of hope for change in the right direction if they can convince the powers that be (ego and all) through all the crimson coloured tape that  these changes are necessary which could lead to the arduous process of modernizing.  The latter company referenced has already made that investment both with full ownership backing down the entire hierarchy of the org chart, and in that case, influenced heavily by yours truly.  I see much greater things for that latter company because it has done the realistic self assessment and see that much is lacking, ultimately choosing to do something aggressive to resolve the situation with a great outlook towards future business.

Hopefully we can see more and more existing companies learn from these examples before they find their up and coming competitors who embrace a method to their madness soaring by and taking their bread and butter in the process.

Rediscovering an old friend, Perl.

It has been over half a decade since I was paid to code in Perl, having since moved on to Python amongst other languages.  I use the past tense because this is all changing once again.  Due to circumstances that are almost entirely personal-life related, I find myself moving on from Yorn after the past fifteen months of doing great things with wonderful, intelligent and foward-thinking people.

After months over two months of screenings, phone interviews and an on-site meeting, I have accepted an offer as Production Lead Engineer at Fanatics, LLC at their Conshohocken, PA facility.  I'm going from working in a Python/Django environment for the past half decade plus into a more systems/internal middle-tier/core environment based on an old friend, Perl.   And despite some of the issues which led to this change, I'm wholeheartedly looking forward to it, and here's why.

As we grown our personal knowledge in a specific discipline, in this case programming languages, our perspectives and approaches mature.  When we step away from our comfort zone into some new experience (or language in this case) as I did professional when I moved on to Python as a requirement of Retail Expert Inc, CTO (at the time) Andrew Sernekos, I shifted mentally and learned the way of the Pythonista.  As the years progressed it became the 'one true way' of thinking, which was fine given that my language dictates were in-line with my choices and professional tasks.  I had put aside my TMTOWTDI (there's more than one way to do it) mantra from my Perl Monger/Perl Monkish days of yore.

Refreshing my skills back into the primary language through which more than half of my professional career was built, Perl, has opened my eyes in a way not previously possible.  The memories I had of Perl with its heavily flexible manner/varieties of expression had originally left a negative taste in my mouth but now having disciplined myself cutting through Python, Groovy, Scala and others has proven to expand my view of what Perl truly holds in terms of power.  With great power comes great responsibility and now with the modules from CPAN such as Moose and its derivatives (as an example), the future is looking quite fascinating again for me with a language that has been around now for 35 years.

Here's to the joys of the future and best wishes for the incredible team at Yorn as a sail on to my next contractual adventure.

Theorising vs. Doing

Over the past many years I've found myself slowly moving (in my personal projects) from the realm of actually implementing code about which I've been curious to spending my overwhelming majority of my free time to theorising about various concepts and/or interests, primarily in the field of human language parsing, simulation and simplification of generic real-world object modelling.  Several of these have been mentioned and explored in previous posts over the last 7 years.  

I believe a great reason for this change in focus is partially age, and primarily due to the extent that my days are focused on building the next great startup application.  We recently (as of the New Year) pushed the past eleven months worth of work into existence as our new primary (Python/Django) product replacing the previous multiple iterations of our legacy (Java) product.  The focus and scope of the project limited my ability to go off on tangents wherein I could code my personal projects freely.  This was by no legal hand binding, it was simply a matter of wanting to focus my writing to building our product to the level it needs to be, to the high standards we require (and rightfully so).  

This leave one (me, namely) with little actual time to thunk down in front of my primary workstation to commit theory to codebase, hence I've found myself working it out on handheld whiteboards, sketchpads,  napkins, chalk on the ground whilst playing with my daughter or even simply working these theories out in my head.  While I would like in many cases to put my thoughts into actual runnable logic, I've found that the exercise of stepping through theoretical code in my mind has kept me sharper regarding my thought processes.  True, I cannot share this as easily with friends afar, but given that many of my personal friends are also senior level engineers, discussing (when we do) casually my theories and ideas, they get the gist quickly.  

The point I'm trying to get across here is that while today is perceived as one wherein a bunch of 20 year olds either in or fresh-out-of college spend an obscene amount of hours in incubators or at startups killing themselves churning out ridiculous amounts of code, there is most definitely no slow down in the thinking processes of the more senior of us out there.  We just don't find a necessity it raw churning out of code as it is the concepts that drive the field and aid in future innovation.  Let those who wish to "do" continue to run their path, but lets not overlook the value of those who've moved their focus (whether by situation or by intention) to the more abstract realm of theory as the two are inextricably bound.  We need ensure that these two distinct groups of people are in contact with one another because it will ultimately lead to the newer advances, bettering our field for all involved.

New environments (mentally) as it were... (a.k.a. something old, something new)

Since my last update made a little over a month and a half ago, much has changed.  I've accepted a position as Lead Software Engineer for Yorn, LLC located in Conshohocken, PA.  I haven't abandoned my prior position with the publishing house in NYC, merely time-shifted those hours elsewhere.  Taking this new venture (on top of my previous work, plus the other tech-startup of which I'm a co-founder, Ecquire has proven to be a necessity for my sanity.

Working primarily for the past many years as the sole Software Engineer for a half dozen magazines in a Django/Python/Javascript/CSS/HTML/FreeBSD Unix environment whilst also wearing the SysAdmin hat has been...  draining.  This wasn't due to having nothing to do, this was more to do with having very little room for innovation as the end product wasn't the new SaaS website or Outlook plugin, it was simply repetition with the only real variances being that as dictated by wet-behind the ears designers and graphic artists whose concerns about a given site was more about look than UX.  Don't get me wrong, some of the designs have been gorgeous, but not without the pendantic whining common with individuals who only knowing Adobe Photoshop aren't happy when you explain to them that there is a different methodology at work in producing like-layouts im a web framework.

Either way, back on topic.  I feel for the first time in quite a while that not only am I solving real problems that pertain to a large scale of individuals and companies (as opposed to readership by fashion and/or tattooing enthusiasts) but I am finally in a position in which I have a cohort in crime, a partner, a colleague. Best of all, he's brilliant.  A proper individual for the field, something when I've been trying to find for ages.  A PhD in computer science and a quarter century in the field along with all of the traditionally odd hobbies and sense of humour found more commonly in those with very intellectual professions.  Even though we've only met face to face on three occasions, we've shared many phone calls and Skype sessions whilst working out our designs and product development and it has simply been a breath of fresh air for me.  I have someone off of whom I can bounce complex ideas knowing full well that I'm understood, as well as presenting me the opportunity to expand my own learning horizons.

Excitement stirs ones involvement in a given project and considering that this is what has (and is) happening to me, it is paying off in not only a clearer mind and what I believe are more robust ideas emminating from my thoughts but it has reduced my stress allowing me to be more productive all around whether in the publishing work, my own startup and more importantly in my family life.  It all comes back to ensuring that whatever one does, they need to sometimes step of a comfort zone in pursuit of that which provides incentive and drive in ones given interests and/or profession(s).  I was too conservative earlier on in my professional career and I believe I would've benefited greatly had I stepped out of my cocoon years earlier rather than (as most introverted engineering types do) not think highly enough of myself to be worth more than an abysmal income and working environments to which I was subjected for over seven years.

Times have changed and the future is looking brighter and brighter.  Ecquire & Yorn have paved the way to a better future.  I stepped past my initial nervous/worried leanings and jumped into the world of proper tech startups finding that the grass truly has shown to indeed be greener on the other side.  Now, onto another exciting and challenging day of practising my craft.