SOA Test Approach : The Purpose and How To Do It

Testing SOA could be viewed as a complex computing problem. With any complex problem,  the key  is  to break  it down  into smaller, more manageable components and build quality  into  these deliverables.

The foundations  to successful SOA  testing are as  follows:

• Equal weighting of  testing effort  throughout  the project  life cycle. Many organizations still  fail  to recognize the real benefits of static and formal review techniques during the early stages of the project. Most or all of  the  testing effort comes  too  late at  the end of  the project  life cycle. More  testing effort will be  required at a service  (program)  level.
• The SOA test team  is a blend of business domain and  technology experts.
• Design  the  project  test  approach  alongside  the  project  business  and  technical  requirements. Budget  for  the Test  team  to be  involved  from  the start of  the project.
• Implement Quality Controls  throughout  the project  life cycle.
• Security Testing  is not an end of project activity! Design and Plan Security  testing  from  the start of  the
  project.
• Test  tools are a must!

How do you test SOA architecture? You don’t. Instead, you learn how to break down the architecture to its component parts, working  from  the most primitive  to  the most sophisticated,  testing each component, then the integration of the holistic architecture. In other words, you have to divide the architecture into domains,  such  as  services,  security,  and  governance  and  test  each  domain  separately  using  the recommended approach and  tools.

SOA is loosely coupled with complex interdependencies and a SOA testing approach must follow the same pattern.

Figure  1  represents  a model  of  SOA  components  and  how  they’re  interrelated.  The  Test  team designing  the Project Test approach and plans must have a macro understanding of how all of  the components work  independently and collectively.

You can categorize SOA testing into  the  following phases:

• Governance Testing
• Service-component-level  testing
• Service-level  testing
• Integration-level  testing
• Process/Orchestration-level  testing
• System-level  testing
• Security Testing

Governance Testing

SOA Governance is a key factor in the success of any SOA Implementation. It is also the most ‘loosely’ used term, as it covers the entire lifecycle of SOA Implementation – from design to run time to ongoing maintenance. SOA Governance refers to the Standards and Policies that govern the design, build and implementation of a SOA solution and  the Policies  that must be enforced during  runtime.
Organizations must have well defined Design, Development, Testing and Security Standards that will guide and direct SOA implementations. Quality controls and reviews must be implemented throughout the entire Project  life cycle  to and processes,  to ensure compliance. The appropriate peers must conduct these  reviews and deviations  from  recommended  standards must be agreed by  the organization’s Governance  team.

The  following are examples of SOA Governance Policy  types:

• Quality of Service policies on Performance, Security and Transactions
• Regulatory policies – Sarbanes-Oxley
• Business policies –  rules
• Audit policies – what events need  to be  logged, how  long must events be kept, etc
• Infrastructure policies – access, backups, disaster  recovery and  failover

Test cases will be constructed and executed  in all of  the project  test phases  to determine  if SOA Policies are being enforced. SOA policies can be enforced at runtime, by using technologies and/or monitoring tools.
SOA Governance  testing will not be a separate  test phase. Testing  that SOA Governance  is enforced will take place  throughout  the project  life cycle,  through  formal peer reviews and different  test scenarios  that will be executed during  the separate  test phases.

Service-component-level Testing

Service-component-level  testing or Unit  testing,  is normally performed by  the developers  to  test  that  the code not only successfully compiles, but  the basic  functionality of  the components and  functions within a service are working as specified.
The  primary  goal  of  Component  testing  is  to  take  the  smallest  piece  of  testable  software  in  the application, isolate it from the remainder of the code, and determine whether it behaves exactly as you expect. Each Component  is  tested separately before  integrating  it  into a service or services.

The  following quality and  test activities are  recommended  in  this phase/level of  testing:

• Formal peer reviews of the code to ensure it complies with organization standards and to identifyany potential performance and security defects or weaknesses
• Quality entry and exit criteria are not only defined for this level of testing, but are achieved before
  moving  to  the next  level of  testing

Service-level Testing

Service testing will be the most important test level/phase within your SOA Test approach. Today, many organizations build a program or Web service, perform limited unit testing and accelerate its delivery to the integration test phase, to allow the test team to evaluate its quality. Service reuse will demand each service is delivered from this level/phase of testing with a comprehensive statement of quality and even a Guarantee!

The  following quality and  test activities are  recommended  in  this phase/level of  testing:

• Formal peer reviews of the code to ensure it complies with organization standards and to identifyany potential  interoperability, performance and security defects or weaknesses
• Functional, performance and security regression suites  to be executed against  the service. This will require  the help of automated  test  tools and  the development of sophisticated harnesses and stubs
• Quality entry and exit criteria are not only defined for this level of testing, but are achieved before delivering  the service  to  the next  level of  testing

Service Level testing must ensure that the service is not only meeting the requirements of the current project, but more importantly, is still meeting the business and operational requirements of the other processes  that are using  that service.

Integration-level Testing

The  integration  test phase will  focus on service  interfaces. This  test phase aims  to determine  if  interface behaviour and information sharing between the services, are working as specified. The test team will ensure that all the services delivered to this test phase comply with the defined interface definition, in terms of standards, format and data validation. Integration testing test scenarios should also ‘work’ the layers of communications, the network protocols. This test phase may include testing external services to your organization.

Process/Orchestration-level Testing

Process/Orchestration testing ensures services are operating collectively as specified. This phase of testing  would  cover  business  logic,  sequencing,  exception  handling  and  process  decomposition (including service and process  reuse).

System-level Testing

System Level testing will form the majority, if not all of the User Acceptance Test phase. This test phase will test that the SOA technical solution has delivered the defined business requirements and has met the defined business acceptance criteria. To ensure that this phase/level of testing is targeting only the key business scenarios of  the solution,  the business stakeholders and  testers must  fully understand  the quality and  test coverage  that has been achieved  in previous  test phases.

Security Testing

As SOA evolves and grows within your organization, the profile and necessity of Security testing will increase. Today, many organizations perform an inadequate amount of penetration testing at the very end of a project. SOA combined with Government and Regulatory compliance, will require Security testing activities to be incorporated into the entire project life cycle.

Tags: Governance Testing, Integration-level Testing, Process/Orchestration-level Testing, Security Testing, Service-component-level Testing, Service-level Testing, Service-Oriented Architecture, SOA, SOA Test, SOA Test Methodology, SOA Testing, SOA Testing Approach, SOA Tutorial, System-level Testing