VS 2010 Coded UI Test (Automation Testing) – Challenges

This is core technical article targeting the testers and others please skip

We started testing automation a month ago using VS 2010 and faced some challenges while implementing it. Automation testing is basically a record and playback. That means recording testing activities such as typing URL, clicking button, navigation etc and then verifying in the playback mode. This can be done in 2 ways Web UI Test and Coded UI Test. Coded UI Testing is better since it will record all actions including scripts (JavaScript/VB script).

There are few good articles in basic steps to test using Coded UI test and it’s good to start with. 
(http://www.dotnetfunda.com/articles/article1241-coded-ui-test-basic-walkthrough.aspx)
Those were all good till it’s applied in real time applications with lots of controls, popup windows, AJAX etc and soon found out that very little help is available online. The reason is people have just started using this instead of other automation tools like Winrunner. 

The basic idea behind this article is to list the challenges faced in which some of them are still vague.

1) Parameterising Radio-button list (iteration)

CUI Test was okay for controls like Text box, Dropdown, Buttons etc. In the background, the record engine considers each control as window and they will be added to the list of controls to be validated while playback.

The radio button list will be rendered as separate radio buttons (with separate ID) in the html and hence only selected option will be added to the playback control list. Now here’s the problem – there’s a RB list with 5 values and I am clicking 3rd option while recording. A parameter CSV file is created for iteration in which I want to select 4th value and this was not possible since its not available in the list of controls.

 2) Hyperlink with dynamic numbers (iteration)

There’s grid which shows list of records (say list of employees) and the 1st column is a hyperlink which shows employee ID to edit the record. The recording is performed by searching for employee, clicking a hyperlink; editing the employee and saving it (say record ID 10). Now iterate this test using CSV file with some other record ID (say 15). This was throwing error since the control ID will be stored based on the ID selected and it will not exist in the playback.

Solution – Permanent solution was not found. So did a workaround to add key press event for TAB in the code file.

SendKeys.sendWait(“{TAB}”);
SendKeys.sendWait(“{ENTER}”);

3) Controls with multiple values (Iteration)

We use CSV file with each values separated by ‘,’ for each controls and ‘Enter’ key for next iteration. This will work for controls which have single value (Textbox, Dropdown etc) and created a problem for controls like List-box and checkbox.

4) Action takes longer duration

Let’s take the same scenario listed in 2 and assume it takes 1-2 seconds to search for records. In the playback mode, it might take longer duration than the recording and second action might overlap (say editing). This will throw run-time error since the control is not ready yet (window not found exception)

Solution (available in MSDN) - There’s a property to wait for completion of each action and this might solve the issue (can set for entire test). But the playback becomes very slow and it was taking twice the time.

// Set the playback to wait for all threads to finish
Playback.PlaybackSettings.WaitForReadyLevel = WaitForReadyLevel.AllThreads; 
// Press the submit button
this.UIMap.ClickSubmit(); 
// Reset the playback to wait only for the UI thread to finish
Playback.PlaybackSettings.WaitForReadyLevel = WaitForReadyLevel.UIThreadOnly;

5) CUI Test in Virtual environment

We tried CUI Test in virtual VMWare machine with Widows 7 OS using VS 2010. But the actions are not even recorded in virtual environment though the system has all specified configuration. It was strange since the virtual machine should behave like physical theoretically. We could not find any online help why the actions were not recorded. 

Solution – Finally we solved the issue by migrating to virtual machines with writable storage :- C drive will not be reloaded on new session. But the bottleneck in this approach is that our company has to pay lot of extra money since writable VM storage is costlier than read-only one.

6) Recording more than 1 modal popup window

This was another issue. We have 2 modal window to perform some actions (say adding department, add employee) in one screen. The funny thing is, it will work for 1st window and throws error while playback for 2nd window. The playback will wait for a long time and throws exception saying the control is not found or cannot perform the action.

Solution (available in MSDN) – When UI Test framework searches for second window; it will search for cached reference and will fail. There’s a workaround to find the window

this.UIDialogABCDWindow.Find();

This code should be added in the code file (UIMAP.cs) before action on the second window (Close/OK)

7) Schedule CUI Test script

It’s not always possible and needed to use Visual Studio for running Coded UI Test code. In practical scenario, the test should run automatically and give results on a regular interval (say daily once). Coded UI test will generate a DLL as output and this should be used in the Scheduled task

Solution – There’s a command line option to run CUI test and this can be added in BAT file, which can be used in Scheduled task. 

Cd C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE (path of VS 2010)
mstest /testcontainer:D:\TestRun\CUTest.dll (path of CUI Test output Dll)

Note –  The above command can be used to execute all tests created in the project (sorted by name) and to specify particular test use"/testcontainer:D:\TestRun\CUTest.MyTest1". Command "mstest -help" will list all command line arguments. Also the screen should not be locked and screen saver should be turned off to run scheduled tests without error.

The End

Above are the challenges faced while using VS 2010 for automation testing.  If you have found a better solution or better ways to solve them, you are welcome to add comments which might help in future. Happy testing..!

Comments

  1. Hurray !!! Anyone can add comments from today !!

    ReplyDelete

Post a Comment

Please add your valuable comments