11 April 2012
The code reuse myth: why internal software reuse initiatives tend to fail
Despite all the best intentions, many software reuse initiatives fail to deliver on their promises. Code reuse rarely becomes an integral part of the development process and is often confined to third party frameworks and products. Are we expecting too much from software reuse and should we learn to set our sights a little lower?
It's more difficult than it looks
The largest single barrier to effective code reuse is that it is difficult. It doesn't just happen by accident. Reusable code has to be specifically designed for a generalised purpose and it is unlikely to appear spontaneously as a natural by-product of development projects.
Reusable components are usually designed to serve an abstract purpose and this absence of tangible requirements can make them unusually difficult to design, develop and test. Their development requires specific skills and knowledge of design patterns that is not commonly found in development teams. Developing for reuse is an art in itself and it takes experience to get the level of abstraction right without making components too specific for general use or too generalised to add any real value.
These design challenges can be exasperated by organisational issues in larger and more diffused development environments. If you are going to develop common components then you will need a very deep understanding of a range of constantly evolving requirements. As the number of projects and teams involved in reuse grow it can be increasingly difficult to keep track of these and assert any meaningful consistency.
Successful code reuse needs continuous effort to evolve shared assets in step with the changing business and technical landscape. This demands ownership and governance to ensure that assets don't fall into disrepair after the initial burst of effort that established them. It also requires a mature development process that provides sufficient time to design, test, maintain and enhance reusable assets. Above all, you need a team of skilled architects and developers who are sufficiently motivated and empowered to take a lead in implementing code reuse.
The folly of framework-led development
Many code reuse initiatives focus on the development of a re-usable framework of components that can be integrated into individual applications. This is an obvious starting point, particularly given that framework-based technologies such as .NET and J2EE have gained mainstream acceptance over the last fifteen years.
However, framework development should be treated as a separate activity to normal application development. It is a mistake to try to lead application development from a framework perspective as most application-derived frameworks have very little value beyond the precise project that they were written for. Unless you are really focused on exactly what your framework is supposed to be delivering then you can end up wasting time generalising single-use functionality.
It can be difficult to clearly define the scope of a component framework. It is important to bear in mind that applications are unique - or at least should be. Many requirements are best served through custom development for a specific use case rather than shoe-horning in code that was designed for a related purpose. A framework-led approach to application development can undermine the optimisation of an application and encourage grand concepts over simpler solutions.
Many code reuse initiatives flounder because they try to achieve too much too quickly. Grand notions of an internal framework that accelerates development may be seductive, but something less ambitious may be easier to achieve and far more practical.
Who is paying for this?
Securing budgetary support is often a barrier to effective code reuse in a commercial development environment. Most software development is funded to meet immediate business objectives and this can give rise to an overtly short-term focus. It can be difficult to build a business case for reuse activity that has long-term benefits that can be difficult to quantify in terms of precise return on investment.
This difficulty is compounded when development is funded on a departmental or client-specific basis. This narrow base for funding undermines business-wide initiatives and many organisations find it difficult to arrive at charge-back or group funding schemes to pay for code reuse initiatives. Given that code reuse does not emerge spontaneously from on-going development work, it can be very difficult to secure the necessary funding to deliver it as a separate on-going development project.
You can take a horse to water...
A reuse initiative has to be accepted by the development community to succeed. You can develop all the frameworks and guidelines you want, but they don't have any value unless the developers use and value them.
There can be political resistance to reuse as developers feel more comfortable when they have full control of the system that they are working on. They can often be suspicious of "top-down" reuse initiatives and resent the restrictions this places upon their freedom to make their own architectural decisions.
Developers have a natural tendency to try and "roll their own" solutions rather than adapting to an external component. This is understandable given that it's easier to work with your own code than figure out something written elsewhere. This cultural resistance to code reuse is often referred to as "not invented here" syndrome and it requires sustained effort to win over hearts and minds and foster a shift towards a culture of reuse.
You need to attract developers towards reuse. It's important to establish "reuse magnets" that provide credible and authoritative repositories and help to foster a sense of community among users. The open source model can be particularly helpful here as it can encourage engagement as well as shortening the feedback loop for bugs and new features.
Hasn't somebody else already written this?
It's wise to consider code reuse in a wider context and embrace external frameworks, restricting the scope of your internal development to those areas that bring something unique to the table. Code that has been written for specific, niche purposes tends to be a better candidate for reuse particularly when it encapsulates intellectual property. Code written to solve more generic problems is far less likely to add value, mainly because somebody else has almost certainly solved the problem before you.
It can be easy to fall into the trap of duplicating functionality that has already been implemented in existing frameworks or open source libraries. This "commoditised" functionality should not be duplicated in the name of code reuse - it's already there for you to use. For example, a pricing model for an investment bank is something unique that you would want to perfect and reuse. A library for implementing messaging is not something you should consider writing yourself - somebody much smarter than you has already solved the problem.
The art of the possible
Code reuse is widely accepted as a "good thing", but it's important to manage your expectations of what it might deliver. You're more likely to be successful if you accept the limitations of your environment and restrict the scope of code reuse to something achievable that can add genuine value and gain widespread acceptance. Instead of developing grand designs for an internal code framework, it's often best to start small, develop iteratively and progressively build on small successes.