So in my last post I talked about my time over that year or so on a development team. Our massive project over that time has been to rebuilding a front office pricing platform for a major bank.
It was a project that had been attempted at least once in the past and intact the system we were rebuilding was basically a rebuild of a system from the 90’s. This was my first major Front Office (as in software directly supporting traders in an investment bank) project and as such I guess I had a certain naivety about how it would go. I have been successful in doing rebuilding tasks in the past in other mathematical based industries.
I have found Front Office presents many of the same problems that have been present in other industries.
One is that business people have no interest in architectural loveliness. This simply want flexible reliable IT that was what they want for the lowest price and wait.
Another is that people get upset when you start saying a system that they support, or maybe even designed, is so bad it should be re-written.
However one, rather large, new problem was that IT is so business led in this area that every business request must be met in the shortest possible time and therefore there is a lack of code ownership in these places. There is central code at the company I am consulting at that can be updated by tens of teams across numerous source control branches around the world. Each team will implement it’s business function requests in anyone of a dozen or more systems. Therefore each system has been worked on by dozens of developers over a fee years and therefore, in most instances, there is no single document that will be able to tell you completely what the system does or why it does it.
An example of this is that I have been working on a project for the last 16 months to replace a body of shared code with a new Market data service. Despite the fact we have been working on the code for so long and we are supposed to be the team that supported the old over, only last week did someone from another team come to us asking if we had taken into account that on a particular few trading desks at particular times of the day that the behaviour of the system differed in a fundamental way. Let me tell you that this came as a shock to me.
Now I understand a that this is an extreme example, but it illustrates a point: despite what you think about how easy a system will be to re-engineer, it will definitely be harder than you think. When you see strange if statements and convoluted logic, it might be bad coding, but it might also be a bug fix for an obscure bug which only happened with the right ‘perfect storm’ conditions, or normally a bit of both. If a system works but is ugly to code on, it might not be worthwhile starting from scratch.
I’m not saying things shouldn’t be re-built, I still think my project over the last while has been totally worth it. The old system was too inflexible, to un-scalable, and too hard to test and debug to continue development on. Not only have we made a new flexible system but we have made it a little faster and a little more memory efficient. We have also started to see the old system fail in production much more frequently recently.
I suppose the moral of this story is that rebuilds in software always sound like more fun then just improving an old system but will always be harder than you think.