17 Mar 2011
Following the 2008 liquidity and credit crisis, investment
banking regulators across the world have continued to introduce
unprecedented restrictions on existing capital adequacy
requirements and associated liquidity risk management
practices.
Different regulators and regulations apply to operations in
different parts of the world. Most major investment banks
have international operations operating in accord with 24x7
markets. Changes to the regulatory framework are in conflict
with the unprecedented speed at which financial institutions are
seeking to maximize their profits, cut the cost of their business
operations and embrace the opportunities presented by changing
technology.
From the board level and trading floors, through to the
compliance and audit departments, the implications of these new
regulations must be addressed. The fast-paced and demanding
trading room environment requires quick fixes to be made to
circumvent the limitations of the existing technology, whilst
ensuring minimal client impact. In contrast, down-stream business
functions require more structured and thorough approaches to ensure
that the banks do not become exposed to malpractice and unwanted
attention from the authorities.
This article considers the internal conflicts and challenges
that investment banks are facing, and how a strategic approach to
testing the required software changes can alleviate some of the
problems being faced.
It should be noted that all investment banks are different in
attitudes, culture, technologies and programme management
techniques. The recommendations discussed here therefore need to be
viewed alongside these other considerations.
Interpreting regulations and mandates
Certain Basel III requirements, such as theExpected Positive
Exposuremandate will dictate software or system changes. The
implications of the software changes upon applications will need to
be investigated and applied across all the impacted systems. The
nature of the new regulatory requirements is such that they require
a complete understanding of both the technology infrastructure and
internal trade processing capabilities to ensure that the changes
are fully understood. As a result, the key to successful
implementations of this nature is to deploy project and testing
teams that can demonstrate domain knowledge that complements the
internal system capabilities of the bank.
One key issue faced by investment bankers is the apparent
conflict between regulations, as yet unresolved by case law. For
example, the Data Protection Act 1998, the Freedom of Information
Act 2000, and the Data Retention (EC Directive) Regulations 2009,
each have different requirements on data retention. Similarly
Sarbanes-Oxley imposes different retention rules for data used in
the US.
System architects and technology teams may not be aware of such
regulatory requirements; their focus is more likely to be on
testing and optimizing the system performance of real-time trading
systems. Latency of an application at one institution may cause an
open trade offer to be taken by a competitor with a better designed
and faster system. However, testers of such applications will
also need to take decisions on compliance, and make provisional and
conditional assessments of fitness for purpose, often pending the
advice from internal compliance and audit departments who may be
unable to provide such advice in the required timescale.
Achieving testing objectives in a rapid delivery
development
It is an accepted fact that the majority of systemic changes
within today's market place are both complex and challenging, and
that meeting the demands of the various business areas places
unwelcome pressure on both project and testing teams. The nature of
the trading room environment tends to attract high profile
individuals that are demanding of themselves, their colleagues and
the supporting areas of the financial organization. Of all the
challenges that are experienced today, one of the most common is
related in some form to the variety of thedifferent personalities
and associated characteristicsof the people that are employed by
the banks.
The majority of change programmes originate from one or maybe
two business departments rather than an entire financial
organization. The trend is that the originating department is
likely to benefit from the change itself, and therefore has a
direct interest in pressing for theTime to Market(implementation
timescales)to be as short aspossible, to ensure that the cost of
change is kept to a minimum. Other departments tend to have an
indirect interest in the change programme, but are likely to view
additional project work as a burden and unnecessary overhead in
terms of both resource utilization and workload.
Whilst traders and front office staff expect rapid delivery, the
reality is that the development and testing of projects requires
detailed planning and scheduling of resources. In achieving this,
it is necessary to engage business managers with sufficient
knowledge to understand the importance of testing, and with
sufficient authority to ensure that it is correctly undertaken.
Such engagement means that testing should be at the forefront of
change programmes to ensure that the requirements are clear and
concise, that impacted business areas are identified and engaged at
the project inception, and ultimately to ensure that testing is
part of the complete development lifecycle.
As most investment banking programme managers favor an agile
methodology to software development, this should mean that
continuous testing is performed throughout. Experience,
however, shows that true test driven development is not fully
understood, and that testing often takes place towards the end of a
project, when deadlines are fast approaching and timescales are cut
short. As a result, insufficient time is made available for a
comprehensive phase of testing, and as testers find defects in the
software, subsequent delays are attributed to the testing teams
rather than the overall project management.
Front office fluidity versus regulatory constraints
Traders and sales personnel do not typically view testing as
part of their day-to-day responsibilities and will often require
coaching or at least guidance on testing best practice. Operations
personnel tend to take a far more pragmatic approach that embraces
the need for a comprehensive testing plan. These different views
can result in a conflict of effort and timescales that result from
the minimal to the extended testing coverage throughout the
lifecycle of a project.
Whilst the majority of business areas that support the trading
systems prefer a risk mitigation policy, the requirement for rapid
system change to support business demand, can result in conflicts
that potentially have damaging implications to both the project and
the testing teams. An example scenario is where a new trade type is
implemented as an exceptional item, bypassing most internal audit
procedures and without proper notification to ancillary back office
accounting and transparency requirements. It will then be
integrated again, as formal due diligence processes sanction the
changes.
In this scenario, the solution is todeploy a strong test manager
with both technical knowledge and a good understanding of testing
best practice. However, the most important quality in these
circumstances is for the test manager to possess strong stakeholder
management techniques.It is always important to engage all
stakeholders at project inception, rather than just those that are
directly impacted by the system or software change.
Technology constraints and considerations
In recent years, the demand for technological innovation has
increased, particularly with the advent of rich internet
applications (RIAs) that dramatically improve usability and access
to bespoke user-data. As a result, technology change is no longer
the preserve of the technology experts, with increasingly
technically astute end-users now demanding more from the
applications which populate their order books and provide them with
market data.
The user experience encountered elsewhere, such as on
smartphones, social networking sites and associated applications
are now setting the bar for user interaction. The testing
implications of this are immense, with new RIAs often providing
multi-layered complexity that requires thorough testing, such as
trading websites that employ Adobe Flex technology to provide
complex solutions for both trading and information
distribution.
The implications of utilizing these emerging technologies means
that testing needs to remain one step ahead and be aligned with
rapid development practices, in order to accommodate consumer
driven technologies. As a result, the following testing
challenges need to be considered:
- Not to be distracted by the 'rich internet' of a RIA; the
front-end is important, but so is the way in which these new
applications hook into legacy systems
- Ensuring the new approaches are technically robust enough to
cope with:
-Security challenges
-Changes to the back-end
-Architecture changes
-Requirements to test hand-held devices (and the associated
security implications)
The development of tools to automate the functional testing of
new technology lags behind the speed of change of the technology
itself. This is attributable to the conflict between the new
technology taking market share, and the longer term development of
tools that evolve to thoroughly test its functionality#, especially
in an enterprise environment. This conflict is evidenced within
investment banking, whereby the development and use of automated
testing tools for Flex from Adobe falls far behind the take-up and
use of Flex itself. Whilst potentially opening new
opportunities for the developers of automated testing tools for
RIAs based on Flex, their adoption within investment banks is
slow.
Much has been written about the migration to 'cloud computing'
and its suitability for testing. Such technology is rarely used
within investment banking, particularly as a result of strict
regulation regarding the confidentiality of banking information.
Whilst some banks are adopting a 'private cloud' approach to some
information and services, this emerging technology is still in its
infancy. Test analysts must be aware of the sensitivities of
bankers to the risk of data loss, and it is a credit to investment
bankers that sensitive information appears more likely to be lost
by public services than by banks. However, in circumstances where
restrictions are not imposed, individual traders are able to use
cloud services without sanction; thereby risking the breach of data
protection and other regulations.
Testing across multiple platforms and technologies does not lend
itself to a traditional end-to-end test strategy, as quite
often new technologies tend to conflict with legacy
systems. For example, where new 'publish and subscribe
message technologies' are employed using specific 'peer-to-peer
authentication', it may be very difficult to identify which
services consume internally broadcast messages or transactions. The
considered solution in this situation is to assume a three-phased
approach:
1. Integrate
Systems integration testing is a complex task in many investment
banks, where transactions and data are shared across numerous
disparate systems. The initial phase of testing is to ensure that
information flows correctly in each direction of the test
environment. Normal methods would include an initial 'smoke test',
which can be executed without directly affecting any of the data
stores. These tests are often automated, with a series of
parameters allowing the smoke tests to be conducted from the same
test framework as the later functional testing. Such smoke
tests can be employed as a quality check to ensure that the correct
instances of systems and applications are communicating with each
other.
The goal is to be assured that neighboring systems are actually
able to communicate correctly. Whilst being no different to
standard integration testing in other domains, there are very
particular demands made in investment banking to ensure that ALL
affected systems form part of the tests, and that test execution
always produces a permanent trail to show where data is modified as
part of the testing.
2. Differentiate
The test approach should be built or chosen for its ability to
demonstrate differences in method. Test automation tools employed
in this approach must differentiate between the key layers and
technologies. Some will address transactions at a message level,
testing key functionality at the transport layer. Others will
address data storage, speed of access and the ability to trace
specific transactions from capture to their effect on the bank's
balance sheet, profit and loss accounts, risk, plus applications
alerting those responsible for disclosure to the appearance of
unusual customer transactions. From the outset, the tools and
testing methods must be designed to accommodate the various needs
of different (and sometimes opposing) parts of the business
operation, providing complete coverage. In addition, the tools must
differentiate between the specific features of the different
technologies that have been used.
3. Mitigate
In order to mitigate associated risk, there are numerous avenues
for risk-based testing. When considering the investment banking
domain, we tend to consider the implications of defects from a
number of risk mitigation aspects including; financial risk,
operational risk and reputational risk. With the very large
financial value often involved in trades, risk mitigation can take
a number of forms in business operations, where speed of response
is a key differentiator of margin. Regulations have slowly adapted
to require more and more liquidity to cover open positions,
measured by the risk of counterparty failure. These are all factors
to be mitigated in business, which must be mirrored in testing.
Changes to business processes and procedures
In addition to software and system changes, the introduction of
Basel III will demand a number of enhancements to business
processes and procedures, including most notably the mechanics that
the banks will need to adopt in order to conform to the changes in
liquidity ratio regulations. This will entail the monitoring of the
bank's exposures to different counterparties and clients.
The consequences of this type of change are more in line with
how investment banks embrace change. Whereas system modifications
ultimately enforce a conscious change, amendments to procedure or
regulation require indirect change that the bank will have to
monitor internally.
Conclusion
A natural conclusion to the majority of the challenges discussed
would be to adopt a heavy-handed management approach to ensure that
mandatory changes are enforced within investment banks. Whilst a
number of financial organizations are likely to take this approach,
this does not tend to embrace best practice, or the requirement to
integrate testing efforts across the organization. The recent
experience of City Index in being fined £490,000 (GBP) for failing
to keep adequate records of transactions demonstrates that, whilst
their technology was fit for purpose at a trading level, the
associated back-end data retention fell short of required
standards. This was a clear failure of both IT and the business in
conforming to explicit regulations.
Testing should be viewed as the critical measure of quality and
project success in any IT project. This is more likely to be
successfully achieved by ensuring that from the outset the testing
function is seen as the pivotal project component that brings IT
and the business together.
About the authors
Paul Boston
Paul is employed as a Principal Consultant within the Rule
Financial testing practice. With over 25 years experience of
working in the City, Paul has previously worked at a number of Tier
1 investment banks, software companies and consultancy firms that
support the Investment banking community in the UK and across
Europe.
Roger Fox
Roger is employed as a Test Analyst within the Rule Financial
testing practice. He has over 25 years experience in IT, beginning
in a development team and progressing through a number of
investment banks and consultancies. His breadth and depth of
experience has led to engagements within the front, middle and back
offices of many Tier 1 investment banks.
About Rule Financial
Since 1997 Rule Financial specialists have been working
alongside their counterparts at the world's leading banks and
trading houses, helping to improve productivity, manage risk, lower
costs, and deliver competitive advantage. Our expertise in the
management of change and risk, project delivery, testing,
compliance, and understanding of complex technology has helped us
build long-term relationships with our clients based upon a solid
track record of success. Rule Financial services a range of global
customers from offices in London, New York, Barcelona and Łódź
(Poland)
