SCM v2, Baseline 2.0 and Authoring Studio Testing

The Microsoft Security Compliance Manager (SCM) provides security configuration recommendations from Microsoft, centralized security baseline management features, a baseline portfolio, customization capabilities, and security baseline export flexibility to accelerate your organization’s ability to efficiently manage the security and compliance process for the most widely used Microsoft products.

Client Description: One of the world's most famous software companies. This company provides a variety of products and services all over the world, generating annual revenue of tens of billions of US dollars.

Field: IT Industry

Project Background:
SCM v2 will ship the below new features:Any SQL instance support;Add a setting;GPO Import;UI Cleanup;Local GPO;Setting library / baselines update;Setting group / MCA management.

Baselines 2.0 will ship the below new features:Collapse EC, SSLF;Adding severity recommendations for each setting;Adding Management Package for each product.

New Baselines 2.0:Internet Explorer 9;Windows Server 2008 R2 SP1;Windows Server 2008 SP2;Windows Server 2003 SP2.

Project Area: Software Testing

Size: 500 people*month

Project Characteristic:
·Customer E2E Scenarios:Comprehensive Customer E2E Test Scenarios;
·Customer Bugs:Fix and regressed all customer Issues within Beta project scope;
·Customer Data: Used customers’ real data in Beta test pass;
·Cross Teams Cooperation: Run E2E testing cross teams;
·Test Optimization: Optimize the testing work item between SCM and Baseline Teams;.
·Test Optimization: Testing in various kinds of Customer Scenarios.

Project Difficulties: SCM is an integrated complex cross multiple components, multiple products and multiple platforms tool. There are many end-to-end cross components and cross products customer scenarios. SCM project scope encompasses all governments and enterprise customer scenarios. Traditional functionality test along will not be sufficient.
In addition to the complexity surrounding end-to-end and customer persona scenarios, test team also met with challenges in communication area. Inspur team lacked sufficient design specs and there were always some communication gaps between PM, DEV and Tester.

Solution: Inspur test team integrated “customer focus” strategies into every aspects of the products engineer process from design to test. SCM test teams (i.e., SCM, Baseline, and Authoring Studio) collaborated with multiple teams and involved all stakeholders to come up with a comprehensive end-to-end customer test scenarios.
Inspur test teams also helped the project team to establish best practices in communication area among PM/DEV/Tester to improve the engineer process and the productivities.

Result: Test team signoff SCM v2 and Baseline 2.0 with an unprecedented QA quality bar. The product is designed with customer requests in mind and tested with customer persona scenarios and real customer data. At Microsoft TechEd2011, SCM tool’s presentation resonated in the profound way with 98% approval rating from the customers. We estimated that approximately at least a net saving of 60% engineering cost to the client for customer bug fixes and customer feature requests along.