Sam Farmer

Growing up I never imagined I would play bass guitar for the Dave Matthews Band. And indeed it never happened!

Choosing the Next Great Thing

There comes a time to replace every programming language or technology. Or even more fun to pick a new one! The following process I originally used for selecting a programming language and subsequently have used for other technology selections. I've found it provides transparency, an opinion on what is best, leads to long lasting decisions and provides multiple inputs while avoiding being a committee made decision.

For this entry I will refer to a programming language replacement.

Why Change?

To make sure there is a legitimate need for change — and not just temporary frustration — come up with a couple reasons and discuss them with a few senior team members. If the outcome of those discussions is either neutral or yes then proceed with the process. In the case of neutral select the current solution for the evaluation.

Set Objective

Establish a clear one sentence objective. For example; Select a back-end language to build our API layer with for the next five years.

Ground Rules

The time it takes to learn a new language pales in comparison to the time it takes to learn domain knowledge about a business or organization. Replacing a team with years of hires and camaraderie is an expensive proposition so keep them on board. Gather the team and lay out why you believe a change is needed, how the process will unfold and the steps you are taking to make it transparent.

Be ready for several different reactions. There will be those who agree wholeheartedly (and have been pushing for change for a while) and may exclaim great happiness. Without dampening down those emotions too much make it clear that bashing the existing solution is neither welcome nor productive. And that is because the other reaction will be anger. There will be those who still believe in the current solution for logical or emotional reasons. Many on your team have supported themselves, invested years in expanding their skills and now may feel you are throwing that away. Make it clear that their domain knowledge is more important, that there will be a path for training and, if need be, agree to place the current language on the evaluation list.

The other reaction will be those who feel neutral. They may genuinely see both sides, feel ambivalent about the language or not care. I'd worry about the later and set up some time to talk about it.

Identify other teams who need to have input or be informed and have appropriate discussions.

At the end of each stage below inform the teams of what the stage achieved and what is next.

Evaluation Steps

The evaluation process is set up to first go broad and then deep. Broadness ensures that all options will be looked at and demonstrates this will be a fair process. Going deep provides enough time with each option to understand its capabilities. This is achieved by having a Must Have and Objective step.

Make a list of criteria for the new programming language and start to bucket similar items into, well, buckets. Review the list and mark those that are must have capabilities. An item might be both a must have and an objective criteria, for instance, "IDE Support" in the must have step determines it exists and in the objective step determines the quality.

Make the criteria available to the team for feedback, listen and make any needed questions. Solicit the team for options to evaluate.

Must Have

Phrase all must have criteria so that they are Boolean questions. Gather options and start evaluating. Each option should take between 30 minutes to 2 hours to evaluate and is generally a fun exercise as you get to learn about something new.

Those options that have a yes for everything make it to the objective step.

Ideally there are two to four options left. If more take another look at the objective criteria for must haves or based on the knowledge gained so far in the evaluation determine a new criteria. Having one option remaining should not make it an automatic winner as the objective step digs deep. Evaluate the must have's and see what would need to be removed to have more options. Discuss with your team if those should happen.

Objective

Now is the time for some debates to establish weightings. For each bucket of criteria determine the most important and weigh it a ten. Then rank the rest. Repeat for each bucket. When scoring use a range of 0 to 10 for every item. The combination of score and weighting separates rating from importance while giving an opinion on what matters most. The formula for an individual criteria is Score * Ranking = Total Score.

The objective evaluation needs to go beyond reading documentation and get to know each language. Establish a proof of concept application and build out one for each language. If there are a team of evaluators split out the languages. When writing these applications try to hit a sweet spot between following a tutorial and breaking off to gain deeper knowledge while also not going down rabbit holes.

Hide the weighting and total columns and score the applications against the criteria. Unhide the columns and you have a winner!

Well, not so fast…there is one important stage left.

Assess the Risks

The Objective score gives a mathematical winner. But every decision has risks and evaluating risks sits more in the gut than brain. Assessing risks can take time and I suggest taking the top choice and evaluating if you can live with the risks. For each risk assign a probability (High, Medium, Low), detail the outcome and assign an impact (High, Medium, Low).

Have a meeting with your boss or sponsor of the evaluation and walk them through the risks detailing the impacts. An option here is to outline what a contingency plan to deal with a risk would look like. Depending on the type of decision you may want this person to explicitly sign off on any High or Medium from above.

If the risks are not acceptable remove that option and assess risks for the second choice. If no option remains start all over.

Other Steps

When using this process to evaluate vendors I have often added a performance evaluation stage. Performance may be part of the objective stage often based on vendor provided metrics. This performance evaluation would be tailored to how you are going to use the product.

Wrap Up

The steps are more important than the details and you may want to adjust those to the climate you are in.

I've found this process achieves transparency, looks at a broad range of options, goes deep enough to make an informed opinion and knows the risks.

 

January, 2016