A FTSE 100 company, operating in over 140 countries, had embarked on a global Microsoft Dynamics AX 2012 implementation. Adopting the traditional, partner led, delivery model they invested in an initial template development and roll out to two pilot markets. There were a lot of lessons learnt from this exercise.
With over four hundred and fifty thousand lines of customised code this was a software development project and not a package roll-out. There was no testing team sitting between the partners development team and the business, the system went straight into business testing. The business raised hundreds of defects, a mixture of basic software errors and mismatch between what was needed and what the team had been aiming to build. The business were ill equipped to test, taking much guidance from the development team, and this led to ineffective testing. Once live more issues emerged and a long period of stabilisation was required putting the overall programme back years.
Ultimately the organisation took stock. They brought in fresh leadership. Leadership that had enterprise scale experience from the wider, non AX, world. A consequence was a recognition of the need to move away from a partner led model. To separate out design, build, tests and deployment and to have separate expertise led by the organisation itself working in each area. In line with best practice for large scale programme this led to the sourcing of a separate autonomous testing capability.
Our capabilities, as suppliers of specialist test consultancy and test management services, were known to a number of the new leadership team. We were asked to undertake a review of the programme looking not only at testing but also at the approach to development. We were asked to propose development changes that would reduce the risk of major delays due to late discovery of defects and quality issues. We were asked to identify how to approach testing. Following the review we were then asked to provide not only the independent test leadership but also the test delivery capability that the programme needed.
Working to very tight timescales, driven by the business cycle and the need to get the major US market live within an acceptable timeframe, we faced major challenges:
Mobilising at scale we, within three and a half months, tackled these challenges. The team:
Unlike previous releases business testing, at the end of the programme, did not find significant issues with the AX solution. Defects had been found and eliminated by our independent test team rather than remaining, hidden, in the system waiting to frustrate the business users. Testing by the live businesses was very clean and most of the issues raised by the US business related to data and previously unrecognised requirements. The benefits of having a disciplined autonomous test team were clearly seen in this outcome.
What made this release different? Many things including: