Test Studio and ASP.Net

Background

As part of an important project (> 3000 man/days) developed in ASP.Net and interfaced with the Autonomy search tool, our goal was to automate regression tests.

The application contained many very specific screens and rich graphical components (mostly from the Telerik libraries, from same software editor as Test Studio).

The application also needed to handle many external files, mostly PDF, Word and Excel files. The application was also based on a set of shared databases with existing applications, to which we had no access for privacy reasons.

Technical environment

This was essentially a Microsoft environment with developers running Windows 7, Visual Studio 2010 and SQL Server 2008.

The test servers and integration servers were virtual servers, composed of several servers for application, database and search engine Autonomy. These servers also simulated the customer environment with a dedicated AD.

The tests were run on Internet Explorer 7.0.

Continuous integration was managed by CruiseControl.NET.

Setup phase

The tools setup did not cause any particular problem, as all were managed from within Visual Studio.

The Test Studio tool was still in its infancy, but the Telerik support enabled us to quickly resolve or work around the issues.

We mainly had to find ways to manage the many technical interfaces.

Results

Automation was particularly difficult because of the large number of interfaces to databases (not always directly available) or external files. We had to put in place “stubs” and find tricks to validate the correct operation of the system.

The test coverage was satisfactory for the user interface portion, but incomplete for certain processes that are not directly accessible to the user.

Running the regression tests therefore still leaves a large part that needs human intervention, mostly however to check the results in files or by email.