You are here

Test Automation Tool showdown. TestComplete, HP Functional Tester (QTP), Rational Functional Tester (RFT), CodedUI, White, Telerik Test Studio, Ranorex and Qualiber comparison table

  Test Complete QTP CodedUI Telerik Test Studio Qualiber Rational Functional Tester White Test Automation FX

Ranorex

Watir

PushToTest TestMaker

[Comming soon!]

Canoo

[Comming soon!]

Selenium

[Comming soon!]

Marathon

[Comming soon!]

FEST

[Comming soon!]

Test Management Yes. Built in. No. Needs Quality Center No. Need TFS and MTM Yes. Built in. No  No. Needs RQM. No No Yes. Built in. No No No No No No
Cost About $2K Enterprise Seat License $8K Seat License Free with Microsoft Visual Studio About $2K Enterprise Seat License Open Source Never got to the price. But seems quite expensive Open Source

Free personal edition

License is rather cheap

About $2k Premium License Open Source Open Source Open Source Open Source Open Source Open Source
Separate Test Execution Module Yes. Licensed TestExecute No. Must use Full QTP install Yes Yes No No. That is my best guess. No Yes. Builded in Visual studio with the tests Yes. Runtime License Open Source Open Source Open Source Open Source Open Source Open Source
User Community Big and responsive Biggest by far Small Yes Small Fairly big.  Small A small one Big and fast Fair          
Ease of use Programming background is highly recommended. But with proper planing succeed with resources with very few technical skills. Good for beginners. However, Programming background is needed for advanced automation. But with proper planing succeed with resources with very few technical skills. Programming skills are mandatory Programming background is highly recommended. But with proper planing succeed with resources with very few technical skills. Programming skills are mandatory Programming background is highly recommended. But with proper planing succeed with resources with very few technical skills. Programming skills are mandatory Programming skills are mandatory Easy-to-use tool, no proprietary scripting language - instead real C#/VB.NET; Script-free Testing for Non-Programmers Programming skills are mandatory. However Ruby is quite esy to learn for this purpose          
Support Excellent. SmartBear personnel actively participates in Forums Improving. Knowledge Base access is restricted to Paid subscribers Very little found on MSDN and on the internet Excellent. Telerik personnel actively participates in Forums Very little IBM is know for being not so fast Very little Yes. They have support. Good Support via Support Center (Forum, User Guide, Screencasts, Support Query, Trainings and Webinars) Forums and user groups online.          
Support Cost 20% of Purchase price 20% of Purchase price Expensive support consultants  Not identified if any!  None ? None ? None None          
Scripting Languages VBScript, Jscript, C# Script, C++ Script, Delphi Script VBScript The programming language chosen in the test project in visula Studio Not intended C# & VB.Net Java Scripting. (Not Javascript) and a .Net implimentation. Which I have not tried. C# & VB.Net C# & VB.Net Real C# and VB.NET Ruby          
Version Control Integration Can be integrated into TFS and Visua Studio Via Quality Center only Integrates into Visual Studio Integrates into Visual Studio Integrates into Visual Studio Rational Functional Tester can be integrated with IBM Rational Team Concert as well as with Rational ClearCase®. Integrates into Visual Studio Integrates into Visual Studio Integrates into Visual Studio Yes. It must be possible to write some stuff for that in Ruby          

Descriptive Programming

(Key words, Action words , Primitives)

Yes (Keywords) Very (Action words) No. Solely pure programming. Has a very simple graphical editor for drafted test. But this is not availeble for tests which is not "in draft state" Yes (keywords)  No
 
But you can create you own framework
Yes (keywords) No
 
But you can create you own framework

No. But the code is very nicely structured.

 
But you can create you own framework
Yes (Keywords)

No.

But you can create you own framework

         
Web Testing Yes Yes. IE (good), Firefox (limited) Yes. (IE for sure) Intention of the tool Yes Yes No Yes Yes, all popular browsers (IE, Firefox, Safari, Chrome)

Yes

Whilst Watir supports only Internet Explorer on Windows, Watir-WebDriver supports Chrome, Firefox, Internet Explorer, Opera and also running in headless mode (HTMLUnit).

         
Ajax Yes Yes No! Yes No Yes No ? Yes Yes          
Manual Testing Yes No. It is Quality Center functionality No! Yes No No. That is done in RQM (Rational Quality Manager) No No No No          
Web Load/Performance Testing

Yes (Not true performancetesting)

Performancetesting is better done with Loadcomplete

No. This is LoadRunner functionality Not true performancetesting Yes (Some. But not a true performance tool) No No. But availeble in the Platform No No No No          
Web Services Testing Yes Yes No. But you can develop anything within Visual Studio, if you have the developer skills.

Yes

But requires developer skills. See link: http://www.telerik.com/automated-testing-tools/support/documentation/user-guide/code-samples/general/invoke-web-service-call.aspx

No Yes No No Yes You can write some stuff in Ruby          
Unit Testing Integration DUnit, JUnit, NUnit, MSTest No. It can be done only in Quality Center. Yes. it can run with normal unit test in Visual Studio No No JUnit No No Yes, integrates into Visual Studio, etc. Not intended          
Windows GUI Yes Yes Yes No Yes Yes Yes Yes Yes No          
.NET testing Yes Yes Yes Web Yes Yes Yes Yes Yes Web          
WPF Yes (good) Yes Yes (has issues mapping objects) Web Yes Not very rich applications Yes Yes Yes Web          
SilverLight Yes (good) Yes  Yes (has issues mapping objects) Yes Yes Not very rich applications Yes Yes Yes No          
Flex Yes (good) Yes Yes Yes Yes Not very rich applications No No Yes No          
Java Yes (good) No No No Yes Yes No No Yes No          
QT Yes (good) No No No No Could not find it No No Yes No          
Delphi Testing Yes Yes No No No Yes No No Yes No          
VB Testing Yes Yes Yes No Yes Yes Yes No Yes No          
C++ Testing Yes Yes No No No Yes No No Yes No          
Setup and configuration of test creation Yes. Very good Yes Very Limited. Some can be done. And some can be coded Yes. Very good Coded Yes Coded Coded Yes Technical setup. No "true" installer          
Setup and configuration of test execution Yes. Very good Yes. Very good Very Limited. Some can be done. And some can be coded Yes. Very good Coded Yes Coded Coded, build and executeable Yes Technical setup. No "true" installer          
Test Capturing Yes. Very good Yes. Very good Fair Yes. Very good Fair. With support from object spy programs. Yes Fair. With support from object spy programs. OK Yes

Yes.

Browser plugin

         
Object mapping ability Yes. Very good Yes. Very good OK. but depending on the system under test Yes. Very good OK Yes OK OK Yes

Yes

Web objects

         
Object Remapping Yes. Very good Yes. Very good None Yes. Very good None Yes. But WPF is not good None OK Yes.  Yes          
Execution of tests Very Fast and stable Fast and stable Slow and not very stable Fast and Stable Fast Yes, when working Fast Fair Yes. Very good Good          
Test execution planing Yes. Very Good Yes. Very Good Yes. In Visual Studio Or MTM Yes. Very good Coded Yes Coded Coded Yes

Yes

         
Integration in Team Foundation Server (TFS) Yes. Completely integrated. Yes. Some integration via REST interface Yes. Completely integrated. Yes. Completely integrated. Yes No Yes Yes Yes Yes. It must be possible to write some stuff for that in Ruby          
Tool of a tester or a developer Both. Can be used by both testers and developers. Some more advanced test requires coding skills.but manual testers can produce working automated tests. And this is very importent when deciding your tool  CodedUI is a tool for developers. Lack of intuative GUI is a handicap for CodedUI and the bad mapping of objects too. Both. Can be used by both testers and developers. Some more advanced test requires coding skills.but manual testers can produce working automated tests. And this is very importent when deciding your tool  A Tool for a developer Both. Can be used by both testers and developers. Some more advanced test requires coding skills.but manual testers can produce working automated tests. And this is very importent when deciding your tool  A Tool for a developer A Tool for a developer. But hase some good features for capturing and maintaining tests Tool for testers and developers Both. Can be used by both testers and developers. Some more advanced test requires coding skills.but manual testers can produce working automated tests. And this is very importent when deciding your tool           
Quality Center Integration Possible Intended No Yes No Possible No No Yes No          
RQM Integration Possible Possible Possible Possible Possible Possible Possible Possible Possible Possible          
Links for Demo and more http://smartbear.com/products/qa-tools/automated-testing-tools/new-testcomplete-features/introducing-testcomplete-9 http://www8.hp.com/us/en/software-solutions/software.html?compURI=1172122#tab=TAB1 None http://www.telerik.com/automated-testing-tools/

http://sourceforge.net/projects/qaliber/

http://visualstudiogallery.msdn.microsoft.com/b776c7a8-a1e9-4776-845b-f5c76c93bf83

 

http://public.dhe.ibm.com/software/rational/web/datasheets/rft.pdf

http://www.ibm.com/developerworks/downloads/r/rft/

http://www-01.ibm.com/support/knowledgecenter/SSJMXE_8.6.0/com.ibm.ratio...

https://www.ibm.com/developerworks/community/forums/html/forum?id=111111...

http://white.codeplex.com/

http://white.codeplex.com/releases

http://www.codeproject.com/Articles/289028/White-An-UI-Automation-tool-for-windows-applicatio

http://www.scip.be/index.php?Page=ArticlesNET19

http://www.testautomationfx.com/ http://www.ranorex.com/test-automation-tools.html http://watir.com/examples/