XHProf PHP Profiling

Today I set up my development environment so I can use XHProf to profile PHP scripts when needed, and it was pretty easy.

For starters, I use Ubuntu as the operating system for my desktop environment. So while the information below may be helpful, I will not cover any other OS in my descriptions.

XHProf is a PECL package, and can be easily installed by using standard PECL commands.  However, XHProf is still beta and the default settings of PECL will only install stable packages.  Don’t fear, there is a way within PECL to handle this by appending “-beta” to the end of the module name: (also note how I am using sudo to act as the admin user on Ubuntu)

sudo pecl install xhprof-beta

After issuing the command PECL will do its job of installing the PHP module but will not add it to the php.ini, so that must be done manually.  The default location of the php.ini in Ubuntu is at ‘/etc/php5/apache2/php.ini’, so we edit it there:

sudo vi /etc/php5/apache2/php.ini

We add a single line to the end of the php.ini to activate the module:

extension=xhprof.so

Then we restart Apache for the new setting to take affect:

sudo service apache2 restart

At this point we now have the XHProf module in PHP for Apache related calls to PHP.  If we add ‘phpinfo();’ to a PHP file and view it in a browser we see XHProf is now available for use.

NOTE: This does not make XHProf available for CLI activity. (command line run PHP scripts)  We also need to add the extension to the ‘/etc/php5/cli/php.ini’ file as well to make it available via command line PHP.

Now that XHProf is ready to be used I added it to the PHP script I wanted to profile.  Basically this is just a command to kick off the profiling at the beginning of the script, and some commands to save the results at the end of the script.

In the beginning add the following to start recording at the beginning, and adds CPU and memory info to the output:

xhprof_enable(XHPROF_FLAGS_CPU + XHPROF_FLAGS_MEMORY);

Then at the end add the following to halt profiling and then using XHProf utilities create the output:

$xhprof_data = xhprof_disable();
$appNamespace = 'some_namespace_here';
include_once '/usr/share/php/xhprof_lib/utils/xhprof_lib.php';
include_once '/usr/share/php/xhprof_lib/utils/xhprof_runs.php';
 
$xhprof_runs = new XHProfRuns_Default();
$xhprof_runs->save_run($xhprof_data, $appNamespace);

NOTE: XHProf will dump the results in the ‘/tmp’ location on the system named something like {run_id}.{app_namespace}.xhprof (Ex.- 51f384b8cb9f2.some_namespace_here.xhprof), but the contents are not truly human readable.  I recommend using the tools provided by XHProf to help make them viewable in HTML.

How I made XHProf files pretty

I am Lazy a HUGE fan of doing things simple, so to make the XHProf output readable and easier to use I simply created 3 files in my document root of Apache like so:

index.php

include_once '/usr/share/php/xhprof_html/index.php';

callgraph.php

include_once '/usr/share/php/xhprof_html/callgraph.php';

typeahead.php

include_once '/usr/share/php/xhprof_html/typeahead.php';

No sense in reinventing the wheel.  These 3 files simply included the files already existing in the XHProf module, which call the output files directly from the ‘/tmp’ location on my system.

By calling to http://localhost/index.php I am now greeted with a list of links representing the different files output from XHProf to the ‘/tmp’ location as I executed the PHP scripts calling to XHProf. (shown below)

xhprof_files

Now when I click on the individual links I can view the output from XHProf in a friendly HTML format, like below:

xhprof_details

So, there we go.  Enjoy!

Developer advice

As the organizer of the SoFloPHP User Group I am often approached by entry to mid-level developers asking what they can do to advance in their career or become better developers.  Of course I am nowhere near perfect but have been around long enough to get a few bumps and bruises along the way, so below is what I usually share as some pointers:

Note: While some of these items are kind of PHP specific, others may find useful items as well.

  • No self-respecting person should be up at 4:05am sending emails.  Get some sleep. 🙂  It is OK to stay up late once in awhile, but force yourself to get to bed at a decent time (10) each day.  And try to get up early each day also (6 or 7), which will help you get much more out of your days. 😉
    • The myths about developers working all night on caffeine are false.  Yes, it happens sometimes, but it is rare.  Well rested developers learn more, write better code, and get more work done…period!
  • Track your time, and get in the habit of knowing what you did with each hour.  I personally use Hamster religiously, and find that I get much more done each day as a result. (I have it set to nag me every 15 minutes if I have not set an activity.)  If you are not using Linux as your desktop environment I am sure there are time trackers for the other operating systems, find one.
  • Certifications will not actually carry much value on your resume, so I would not make them a main focus.  Sure they do carry some value, but perhaps not in the way you desire.  Achieving a certification is a great personal accomplishment and will make you feel better about yourself, as well as give you bragging rights among developers.  (Most developers tell you they don’t care about certifications, but deep inside they are simply envious.)  While many certifications are not a true gauge of actual knowledge, they do represent a certain amount of skills.  However, I have found that most employers do not even notice certifications.  I am not saying don’t get them. What I am saying is to be aware the actual accomplishment may be different than you perceive.  When I started getting certifications it reinforced, in my own mind, that I knew what I was doing.  That gave me more confidence overall in my jobs, and was still a big “win”.  But do them in your spare time, not as a focus item.
  • Pick an IDE to use and learn it FULLY. I will not recommend one in this post, so explore and find one that fits how you want to work.  Then learn it COMPLETELY, and use it ALWAYS.
    • If an IDE causes you pain, don’t use it any more.  Pick another one.  This tool will be where you spend most of your day, so you should not be forced to spend your time debugging and fixing your IDE.  It should not crash regularly.  You should not dread opening it, instead you should look forward to launching it.
    • Use all parts of your chosen IDE. (FTP, version control, testing, coding, debugging, issue tracking, etc.)
    • Learn the keyboard shortcuts, they will save you time.
    • Just because an IDE is free does not mean it is good.  You should base purchases on value provided, not $$$.
  • Pick a plain text editor, and learn it well.  There are times you just need to do a quick edit, and opening an IDE, creating a project, etc. is just overkill for this.  Again, there are many of these available so I will not recommend a certain one.  Pick what you like best.
    • The best ones come with syntax highlighting.
    • There are some free ones, but don’t be afraid to pay a few bucks for a good one.
  • Pick a pet “full stack” PHP framework to learn, FULLY.  I recommend either Zend Framework 2, Symfony2, or CakePHP 2.* since these 3 are the most common.  But as with an IDE you should learn one COMPLETELY, and use it most of the time.  Each framework has its strengths and weaknesses, so choose one that works best for you.
    • Good frameworks have mechanisms in place where you can add plugins, modules, or helpers in case the framework does not fully support what your trying to do.  But stick to the framework as much as possible.
    • Feel free to write your own framework, but ignore the urge to use it for employers.  As professional developers we owe it to our employers to use more widely available frameworks.  It is just smart business.  It means businesses can find other developers easier, onboard them faster, and train the group more.
  • Always strive to make yourself replaceable.  If you are replaceable you are also promotable, and you can go on vacation pain free.
  • Learn to use GIT for source control, and use it for EVERY project you do no matter how small.  Sure there are other source control products out there, but currently GIT is the way to go.  All it takes is the command ‘git init’ in a directory and you are of and running.  No excuses!
  • Do things publicly so others can see.  Such as github, BitBucket, etc.  I recommend having code in some sort of public place for others to see how you code.  Don’t be shy.  I’ve had other developers provide feedback on code I posted on github, in a constructive way, and it helped me advance my skills.
  • Your LinkedIn profile is your best career tool as a developer.  Tweak it, adjust it, get everyone you can to contribute to it.  Add projects to it, etc. (See “Build your brand” below.)  Don’t connect with everyone who pops up, and be stingy with what recruiters you allow to connect with you.  If someone is not going to help your career in some way, they do not belong in your connections on LinkedIn.
  • Pick up small projects here and there that are NOT urgent, and you can take your time on.  These little projects will afford you a way to learn new things.
  • Get active in the PHP community.  I mean really active.  Sure, it’s OK to be a member of other communities as well, but the PHP community (world-wide as well as local) is what will really “do it” for you. (If you are going to make a career doing PHP.)
  • Give talks at local user groups, blog about your experiences, follow other blogs of good people (phpdeveloper.org is a good place to see activity of PHP community members blogs. Chris Cornutt does a great job at filtering out relevant posts and adds the best of them on this site.)
  • Get somewhat active on Twitter, join IRC channels, travel to a couple of conferences each year and get to know people “doing things”.  Then eventually start submitting talks to the conferences so you can go talk, and have your expenses covered to go to it.
  • Build your “brand”.  By this I mean to say YOU are the product.  Everything you say and do is your offering.  Your name is your “brand”.  Build the reputation carefully, and before you do anything ask yourself, “Will my customers like/buy this?”  If the answer is “yes”, then go for it.  If the answer is “no” re-evaluate.
  • If you are a woman, be careful.  While women are becoming a larger part of the tech community there are still many men who are not used to it yet.  They are jerks, and your feelings will get hurt sometimes in the process.  Learn to ignore them and focus on the good parts as you grow.  KNOW you are going to do great things, and work toward that progress.
  • Learn Linux via command line.  No need to go crazy with this one, but since most web servers are on Linux it is a good idea to have some knowledge in this area.  You should at very least know:
    • Basic vim commands to edit files on the server.
    • Be able to navigate the OS files and directories.
    • Be able to manipulate files on the server. (cut, copy, paste)
  • Spend some time each day on Stackoverflow.  Try to pick a problem someone posted and help them.  “Doing” is the best way to learn, and there are plenty of problems posted to Stackoverflow daily.  This is addictive, so manage your time and limit yourself.  But do it!

Of course there are many more tips, but I wanted to hit on some key items without writing a book on this blog post.  I hope you find this information helpful, and if you can think of some other hints and tips please feel free to share in the comments.

Good luck!!!

How to grow a tech community

I had the unique opportunity last night to attend a local community meetup where local businesses were brainstorming on how they can attract, and keep, technology professionals to the area.  Years ago Florida made a solid investment in technology when they brought a direct pipe to the Internet through the state.  However, most technology professionals still tend to leave Florida rather than stay.  I live in Boca Raton, Florida and the meeting was held close by, so I decided to attend and see what it was all about.

As most know, I am the organizer of the South Florida PHP Users Group and I am passionate about helping the PHP community grow in south Florida.  Over my years as a developer I have noticed the decline of technology in this market, and specifically the PHP community.  It was this that led me to organize a group dedicated to turning this trend around, and and grow the PHP community rather than continue to watch it decline.

The Meeting

 The meeting started with networking where everyone exchanged business cards and talked, then gathered around a HUGE conference table where the meeting began.  I quietly listened as a new co-working facility was announced, and as the idea of a “computer lab” was brought up as a way to help teach new college students learn things they were not getting in their classes.  Then the conversations gravitated to how local universities were not teaching students everything they needed to know.  The topic of how local businesses should use internships to also teach students “on the job”, then retain them as employees later.  One business owner quickly spoke up and said they did not have the time or money to take on such an endeavor. (Hinting at the true problem in the area, but quickly skipped over.)

The conversations highlighting each persons view of the same topics continued, and anything of real substance was not truly mentioned.  I turned to a fellow developer and said, “They really don’t get it, do they?”  His reply was, “Then tell them.”

After listening to this misdirection for almost an hour I could not take any more, and finally broke my silence.  In my opinion, these are the things that companies can do to attract, and keep, technology professionals. Below is what I covered:

Disclaimer

First, let me state that I realize all companies are not bad.  There are many good companies out there who do treat their developers very well.  I also realize there are companies who pay their developers well, and provide an awesome working environment.  And finally, I realize there are companies who treat their developers as professionals and trust their input rather than using them as a commodity.  I also want to go on the record as saying that south Florida is not unique in how the tech community is mistreated, sadly there are many others.

Also, I realize all people have differences.  Therefore some of my recommendations below might not apply to all.  However, I would assert that my recommendations fit the largest percentage and have been shown to work well with multiple teams I have built and managed.

Working Conditions

Technology professionals need a space with little distractions.  It has been shown that distractions severely hinder professionals because it takes a HUGE amount of time for us to get back into the groove. (some estimates put this number at 20 minutes to get back to where we left off)  So why do companies think it is cool to stuff a bunch of developers into a room sitting around a huge table, or multiple tables, where every movement in the room pulls their eyes away from the screen in distraction?  (So, every time someone moves it causes everyone to loses 20 minutes of productivity.  So how flexible are deadlines?  And how many developers can the company afford?)

Do not mix developers with the general population!  The other staff in the office has no reason to bother developers while they are working.  By not providing a “developer area” it literally cuts productivity in half, or less.

It has also been shown that office workers need their “personal space”.  Not only does this make employees comfortable, but it also allows the employee to “mark” their territory and make it their “home”.  Doing this has been shown to create longevity in employment.  If someone feels at home they are less likely to leave.  Allowing someone to create their space gives a perception of “ownership”, and humans do not leave things they own.

On the other hand developers do like to be able to communicate freely, and having them close together makes sense.  However, I have found that providing a personal space with low (waist high) dividers is wonderful.  It gives developers the personal space they need, and also allows for collaboration when needed/wanted.

Remote or Telecommute Workers

Please do not be alarmed, I am not going to say that all developers should work remote.  However, I think that many are capable of doing it and being more productive as a result.  Many smart companies do tend to stick their developer in a side room somewhere and instruct nobody to talk to them.  (See the Working Conditions section above for tips on how to do this well.)  So, does it really matter if this room is on-location?

Also, if a group of developers desks are put in a room, we notice that very little talking actually happens in the room?  Yet developers seem to “know” what each other is thinking?  This is not voodoo magic.  The developers are regularly communicating through IRC chat or instant messages so they do not disturb each other. (See the Working Conditions section above for why distractions are bad.)  So, if they are communicating without talking or even looking at each other, why do they need to all be in the same place to begin with?

Companies that allow remote working are still able to manage pretty well by having regular meetings, where everyone is required to attend.  This is often used when planning a new project or to relay very important events.  The rest of the time they do not really need developers to be “on location”. (Imagine the money saved for office space!)

Sweat Shop

Most developers actually work around 6 hours per day.  Please do not confuse this to be “Most developers do technology things 6 hours per day”.  That would be a lie.  Developers actually put many more hours into technology each day, but only around 6 of it is actually “work”.  No, just because developers are in front of the computer until 8:00pm does not mean they are getting THAT much work done.  The project will not magically get done faster by keeping developers around longer each day.

Also, please do not fall into the old habits of over-working employees to cut cost.  Forcing a group of 4 developers to regularly work an extra hour or two each day, or God forbid weekends, means another developer hire is needed.  Do not over-work developers.  Adding hours to their schedule does not really get more done, and it will ensure developers leave to work elsewhere.

If you are asking, “What are developers doing after this 6 hours of work each day?”  The answer is probably not as bad as we may think, unless they are not treated well.  Usually they are learning new technologies, or reading blogs and communicating with other developers to keep current, and unfortunately looking for new jobs if they are over-worked regularly.

Future Community Hurt By Moving Too Fast

Many graduates and entry level technology people in south Florida have a very tough time finding a job.  Companies in south Florida, and perhaps everywhere, are moving so fast and have very little money to help grow these entry level folks into a contributing member of the tech community.  To companies who already have a few developers, please augment the team by hiring one entry level person.  Trust me when I say the entry level developer will quickly learn and will be contributing very soon, and may even be the best developer later because they will learn things the “right way”.  Companies will get much more than their money’s worth.

Training (I did not mention this, but should have)

Technology never stands still and is constantly on the march.  New things come out every day, and technology professionals work very hard to keep up with all of this information.  Yet companies do not seem to realize how helping technology professionals learn will inevitably help them.  Companies that help developers learn will benefit.  I am amazed at how many companies will not send their people to developer conferences, or other training events happening.  Not only do these conferences teach new things, but it also allows developers to network and build friendships to help them with problems later.  Then these same companies expect their people to somehow “know” these new things.

On the other hand, if there is a “sales” conference going on, many companies jump at the chance to send as many people as they can.  Then the salespeople re-learn the same techniques and practices that have been taught for decades.  Sales technology has essentially not changed for a very long time.  Seems a little backward, yes?

Respected as professionals

Companies are not professional developers, and many of them have no idea of how to develop.  That is OK, because it is not their expertise, so they hire developers to fill that need.  However, it seems that very few companies actually allow developers to provide their professional advice.  Instead of listening to developers for time estimates, functionality tips, testing requirements, and hardware needs, these things are all ignored to meet some fictional perception the company has for building the application.  Of course the result is typically failure of the application, and ultimately failure of the company or startup as well.

That is kind of like seeing a doctor because we’re sick, or a carpenter to build a house, then telling them how to do their job.

Companies, please respect developers professional advice.  Don’t force them to write bad code to meet unrealistically short time requirements, lack of testing, and poor hardware.  If you hired a professional, let them be one.  Insist they be one.

Salary

I saved this for last because while salary is a concern, it is usually not the largest concern for developers.  If a developer is treated professionally, encouraged/supported to attend training and conferences, not over-worked, and given a nice working environment, they generally are more forgiving of a lower than average salary. (within reason)

Speaking on behalf of developers in south Florida, salary is an issue.  Not only do most companies not follow my recommendations above, but they are also the lowest paying companies in the country for technology related jobs. (By around 10%.)  Meanwhile the cost of living in Florida is among the highest.  This has led many technology students graduating from south Florida universities to leave Florida and go to other areas where salaries are higher.

Now I’m not saying companies in south Florida should immediately start paying 10% more to their developers.  It takes time to make that move.  But perhaps companies should pay less attention to salary when hiring, and focus more on job requirements.  They may find their projects get done faster, better, and more professional because of hiring more qualified developers versus cheaper developers.

Closing

As I found in the meeting I attended, companies do not seem to know how to truly attract technology professionals to the area.  So I decided to help them by writing this post.  Of course there are other things that also help attract technology professional, but if the items I mention in this post are not present then they will not stick around.  We can have the best hackathons, the most awesome coworking spaces, computer labs, and the most funded community events.  But if the developers are not happy at work, they will slowly migrate to areas that offer happy work places.

My experience at Code Retreat Miami 2012

This past weekend (Dec. 8th, 2013) I had the great opportunity to experience my first Code Retreat in Miami, for the Global Day of Code Retreat, here is a post about it to help inform others about this wonderful event.  If you have a chance in the future to get to one, it is a “must do”.

The Crowd

I was very surprised there was a large group of Rails developers, and it was nice to see there were a few PHP people from the South Florida PHP Users Group (SoFloPHP) because I had posted the event on the group page.  It’s comical how the Rails folks (I do not refer to them as Ruby developers, because generally they do not know Ruby.) seem to feel their framework makes it the best tool for everything, and completely disregard every other language and framework on the planet to blindly evangelize.  I say “blindly” because Rails users seem to feel the need to push the framework on me by selling wonderful features, as if no other language/framework in the world has it.  (But I digress, that is another subject for another post.)

It was nice to see all of the programming languages represented.  There was PHP, Ruby on Rails, Java, Smalltalk, C#, and even some Python.  This made for a nice mix to view other languages, and how developers of those languages operate.

Overall everyone was friendly and it made for a great day of learning and fun.

What is Code Retreat?

Code Retreat is a single day pair-programming workshop giving a chance to practice Test Driven Development (TDD) while trying to solve a pretty challenging sample application, and follow the 4 rules of simple design.  The sample application is to build Conway’s Game of Life.  I will not try to explain the Game of Life here, so if you are curious you can click the link and learn of it on your own.

For many, the sample application is not an easy one to grasp. I found that all of my pairs kept trying to pre-define the game board size at the beginning of the process, though one of the criteria of the game is to have infinite size. (Even the moderator seemed to be stuck on the concept as well, and claimed that predefining a size was OK, because it is how he did it.  I found this disturbing, but overlooked it.)

Others found that forcing ourselves to write the tests first (TDD) was the hardest part of the event, and that is what it was all about.  The event is all about learning TDD, so it was justifiable that it was the challenging part.  I, for one, was up for the challenge and forced myself to NOT WRITE A SINGLE LINE OF CODE WITHOUT A TEST WRITTEN FIRST TO COVER IT.  Because of this I had 100% code coverage the entire day.

The day is broken down into 45 minute sessions where pairs work together and get as far as they can.  Usually the first 5 minutes are spent as each member of the pair explains how they envision the logic to work out.  Following that the initial tests are built to test the game board, then code is written to satisfy the tests.  Then more tests to populate the board, then more code to satisfy tests.  Then tests to makes sure the 4 rules of the game work, then code to carry them out.  Finally, if you got this far, tests to ensure a new turn is executed, and code to satisfy the test.

One of the parts of the interesting parts of the challenge is that prior to each 45 minutes pairing session the pairs were changed.  Which required that the pairs started over by explaining their vision.  Then at the end of each 45 minute session ALL code was deleted to ensure you start from scratch again.  This included any hand-written notes you took during previous pairings.

Of course, due to the 45 minutes restriction I don’t believe that anyone could truly get the completed application running.  But that is OK.  The purpose of the event is achieved by showing the benefits of pair-programming, using TDD (Test Driven Development), and bringing the various communities together.

Additional requirements

In addition to the rules and steps defined above, there was additional criteria placed upon the teams for each iteration.  The first round had no additional criteria, but each additional round carried a new requirement to follow.  For this event we had the following criteria set for each round respectively:

  1. Team with someone using a different programming language.
  2. 5 lines or less per method/function.
  3. No usage of mouse.  All keyboard shortcuts.
  4. No talking.

The additional requirements made it fun and forced attendees out of their comfort zone, in most cases.  It was interesting to see how, when faced with the difficulties of the additional requirements, it brought the pairs closer together to tackle the obstacle.

Possible Tweaks

While I did find the event awesome, and enabled developers from across the board to experience new things, I would love to do such an event where everyone used the same technologies.  This would enable to see how other developers in our own area of expertise have adjusted their workflow, and allow further learning.

By having a group using the same technologies it would allow juniors and seniors to learn from each other and grow the individual communities.

My Take-away

For me this was an awesome way to force myself to use TDD to develop.  Too often I get in a hurry and just skip right to the code, then come back and write tests later…maybe.  However, I found that by writing tests first during this event I actually got more coding done and ended up with less code overall.  And it also led me to do less back and forth refactoring during coding, that is normally a HUGE time waster.

I also found that pair-programming was very enjoyable, and led to time savings.  The person at the keyboard tended to get more done while the second person was free to think a bit more, rather than being occupied with the manual tasks of writing the code.

In Closing

I enjoyed the event thoroughly, and would love to do it again….today!  If you get a chance to attend one of these events in the future, I highly recommend it.

Clean Development Series: Part 4, Rewrite dilemma

Lately I have been talking about clean application development, and how developers can do a better job of it.  Due to the strong focus on this I decided to write some content from my latest talks into a series of blog posts.  Enabling everyone to reap the benefits of the subject matter, and allow attendees of my talks to refresh the content covered at one of the events I spoke at.

In part 1 of this series we discussed reasons for creating dirty code, and the things that dirty code can cause.  We followed with part 2 by giving a possible scenario of how developers get caught in a situation and start writing dirty code.  Then, in part 3 we covered “smells” that hint to dirty code, and some possible situations that can be a bad “smell” highlighting potential problems.  So, what happens when our code carries many of the bad situation covered in the first 3 parts?

Code that made us proud at one time has become an embarrassment we try to hide, and hide from.  In part 2 the developer was very happy with his work in the beginning, and so was his boss.  They were able to complete requests quickly, and everyone was happy to have such fast turn-around on new features.  Then, little by little the problems of the application started to grow.  The bugs began popping out, a change in one place leads to multiple problems in other areas, and time estimates start growing to compensate as the developers are forced to pad them for the unknown. As things become worse the developers tend to care less and less about the code, and this lack of caring shows in newly created code as problems continue to mount.

Many times the first signs of surrender come when the boss asks if hiring more developers will help get things done faster.  It’s an honest question, and makes perfect sense on paper.  If it takes two or three developers six months to create a new feature surely twice as many could do it in half the time.  So the boss instructs us to start a search for more developers to hire, and we spend two or three months (or more) of our “spare” time searching, evaluating, and interviewing candidates until we finally manage to find a couple.  After all, more resources equal more output…right?


Needless to say we go through a few developers, because all of them are not able to grasp the mess we’ve created.  Others just leave for greener pastures rather than try to figure things out.  Then a couple, either due to desperation or being hard workers, manage to stick around and get to work.  But for some strange reason the time-lines never seem to improve, and it still takes a long time to introduce new features.

The boss is very confused, and not happy, because he thought more resources would allow development to get done faster.  He even went to his boss and asked for permission to augment the team, and fought a good fight to get his way.  Meanwhile the developers, seeing the writing on the wall, start scurrying to provide answers for failing to meet expectation.  Technical debt has caught up to them, and it’s time to pay it off.

We talked briefly in part 1 about “technical debt”, and explained it was similar to buying things on credit.  Using credit to buy things means the debt continues to grow and grow, until we’re forced to deal with it.  Either we pay it off, or we go bankrupt and start fresh.  In relation to coding, if we continue to develop poorly taking the “short way”, our technical debt continues to climb with each bad thing we do.  Eventually we find ourselves in a situation where we either repair the code through refactoring, or we rewrite the application entirely and start fresh.

Unfortunately the first reaction to a poorly written application is usually a resounding “rewrite”.  Developers feel like a weight is lifted from their shoulders, as the manager hears a distant “cha-ching” sounds of money slipping away.  The old application took months, maybe even years, to build and cost a HUGE amount of money.  Developers, hosting, tools, sales, customer support, and the hidden costs of insurance/rent/utilities/furniture to keep it all running add up.  In the managers mind a rewrite means it must all be duplicated, and that’s not far from the truth when we consider the salaries of an entire development team for the time it will take to rewrite the whole application.

Challenging sale

In the past there has been times when I inherited an application full of dirty code and recommended a rewrite to the client.  On the surface it seemed like a good idea, and a win/win situation for both me and the client.  It allowed me to make a little more money, because a rewrite take longer than simply doing updates or creating some new functionality.  It also meant I could create the application “my way”.  Because if it was built using my favorite framework, with my coding standards, and unit tests, it would be so much easier for me to work with.

The customer would benefit from a “better” application, pay for less time on future enhancements because it was “better”, and might even get some performance gains with less bugs from the application.  The downside of course is that it costs the customer a much larger sum of money, and much longer time period, to have a rewrite done.

The rewrite

After much debate the developers may eventually win, and the manager will allow the rewrite.  But only if the team is divided so part will be dedicated to building the new vision while the rest will support the old application.  It only makes sense the developers who have been there the longest should develop the new application, since they understand the business better. (Even though they were the ones who created most of the bad code to start with.)  The newly split up developer teams start working.

Rewrite

As the “experienced” team starts the new application the other team continues to fix bugs, and develops new features in the old application.  It happens this way because business must continue to make money and service customers who continually need new functionality, or they will leave.  Which causes endless scope creep for the team developing the new application.  Not only are they attempting to build a new application, but they must also keep pace with new features added to the old application as well as incorporate any bug fixes and/or changes to the business logic.  We start to see how difficult the task of rewriting becomes.

Rewriting started with the best intentions as everything is planned ahead of time, code written in an organized way, and perhaps unit test are even written.  But as time moves on, and the scope continues to creep, the business side of the company starts to become impatient.  The developers start to hear, “Is it done yet?”, and under mounting pressure to speed things up they fall back into the bad practices used in the old application.  Before long the new application starts to suffer the same technical debt issues, and the downward spiral drags on.  Ultimately the developers end up in the same position of apologizing for padded time-lines, unpredictable bugs, and cascading errors.

Usually by the time this entire process plays out there has been a change of developers, and maybe none of the originals are still around.  The current team is now faced with two similar applications that both contain dirty code full of smells, and may not even have tests.  They are now faced with the same choices of the previous team…to rewrite or refactor.  And if they refactor, which application do they refactor?

Meanwhile the company, waiting for the new application, has been dragging its feet on some new features.  Or perhaps have been promising the new application to their customers and building expectation.  This costs the company money in lost revenue from lack of features, lost customers who went elsewhere to get the new features, and higher costs for the extra developers required for a dual-application path.  Eventually, if left unmonitored, companies cannot recover and go out of business from scenarios like this.

Refactor instead

On the other hand, what if we fought the urge to push a rewrite?  If the current application is working and already contains all the business logic needed, but has a very dirty code base, why should the customer endure the loss of time and money for a rewrite?  Plus, if we are new to the application and don’t really know all of the business logic, why should we endure the headaches and heated meetings while we discover every little nuance of a new application.

Now don’t get me wrong.  I realize there are some circumstances where a rewrite is absolutely necessary.  Perhaps the framework the application uses has had a major upgrade, or perhaps the application business logic is undergoing a large overhaul, or maybe the code is just…that…bad!  Yes, there are times a rewrite is inevitable, and simply MUST happen.  What I am introducing here is the idea that many times when we do a rewrite it may have actually been better for the customer, and maybe us as well, if we had simply performed refactoring as we went.

Refactor

What I have come to realize as a much better solution than the rewrite story above is to follow some of the steps, but take a different turn.  What I mean is that I still split the team up, but rather than instructing one team to build a new application I dub them as the “refactor team”.  Their job is to start refactoring the code one module at a time.  First they write unit tests, if there weren’t any.  Then they start performing scheduled refactors.  Meanwhile the other team starts following the new coding standards and styles, and writing unit tests as they continue to do bug fixes and adding enhancements.

Of course these two teams need to be closely monitored, and must communicate very often to ensure they are not stepping on each other.  IT IS VITAL that everything be managed carefully, and that the teams are communicating open and often.  Did I mention that communication is important, and it must happen often?

Sure there will be times when commits clash, and code will need to be altered using pair programming.  However, the cost of time and money will be greatly reduced from a full rewrite.  Not to mention it will help bring the development team closer together in the process.

After the initial modules are refactored in this manner it may be beneficial to rotate some members around on each team.  This will ensure that team members on both sides of the fence do not become bored, or fall into bad habits.

In closing

Remember, I am not saying one way is better than another in every circumstance.  It’s really a judgement call on which method is used to deal with the technical debt of the application we are working on.  However, I hope this post has helped highlight that a rewrite is not always the best solution.  Sometimes the best approach is to plan a series of refactors to recover the application from the junk pile.  We must carefully analyze the entire picture and determine if a rewrite is a “must”, or is it a “nice to have”.  Then we can honestly create a full list of pros and cons to more fully gain the big picture, and use that list to sell what we believe to be the best solution.

It may very well save the company from being a “used to be”, and instead make it a “will be”.