Learning Selenium/WebDriver

Recently, I am facing a question lot of times.

How do I learn Selenium/Webdriver?

That question is asked by my friends, friends of friends, co-workers who wanted to learn Selenium but couldn't know where to start. I tell them, that is easy, go to youtube.com, search for webdriver talks in Google Tech Talks, Google Test Automation Conf, Selenium Conf etc, you will find videos,presentations by creators of Selenium.

After repeating this lot of times, I thought, Ok let put those video links some where and send it to them. Because lot of them dont know who created Webdriver or commiters to Webdriver. So here are the links. I will keep updating this whenever I found time or new things.

First, Selenium IDE from SeleniumConf

GTAC 2011: WebDriver

Automating Your Browser Based Testing Using WebDriver 

Se Builder

Web performance testing using Webdriver

Apart from those sources, you can also look into these places to find Selenium related things.

Selenium Officical HQ http://seleniumhq.org/
Selenium Blog http://seleniumhq.wordpress.com/

Managing PDF/Documents using Benubird PDF

Having the books, presentations, articles in PDF is the best way to read and use it across platforms. However, if you have more than thousands of them, then it is difficult to manage. Here my account of my efforts to find a way on my own and how I stumbled upon a good product to manage it.

When I started to use PDF books few years ago, at that time, with few files organizing into folders was easy. Then Google Desktop came, that made life easy. But that has other issues such as re-indexing when moving folders/system and of course the security vulnerabilities. So, I decided to go for a primitive solution. Have the name, title, author, description and location of PDFs in a spreadsheet. I made some tinkering so that I can open a pdf from spreadsheet. This too has some issues. If I send the file to someone else they also have to do this.

Ideal solution is to have those properties in the PDF so that extracting, sorting etc would be easier. Alas, it took long time for me to realize. So, how to do that? First, to know, how many files have those properties?

I started to search for python PDF packages and found pyPDF is good in extracting information from PDFs. But if the PDF size is more than 15MB then it hangs. Apart from that, edit the property of the PDF is only possible by creating a new PDF with those properties and content. This means I will end in duplicate files and need to check those files manually before deleting them. I've tried other tools too but it is same thing. Even the famous ExifTool is also creates new file.

So next thing is search for pdf manager. First hit is Mendeley. Actually it is managing your research papers, make the citations easy not for other purposes. So, it doesnt have any option is edit to title in PDF. You can only edit that in Mendeley not outside of it. That is not I am looking for but coming  close.

Next, searching alternatives for Mendeley. After reviewing few apps, found Benubird PDF. To my surprise, it offers easy editing of PDF properties and this app will write that to PDF. But dont try this with PDF opened in Acrobat. Benubird will overwrite the pdf leaving only one page. Ideally that throw a warning but it is okie as of now. It has other nice features as watched folders, smart lists, lists etc. UI shows Filename, title, author, Subject, tags size, type, path etc.

I have transferred all of my spreadsheet data to PDFs via Benubird. Hope that it will work fine. 

Other links if you want to check
PDFMiner

pdfrw 

pdflib

Logging Selenium WebDriver

With Selenium Webdriver version 2.15.0 logging is enabled to help the debugging. By default logging is enabled. Just add a logger to get the logs.

Setting log level is also easy. Set in the logger or use this code

((RemoteWebDriver) driver).setLogLevel(ALL);

As this helps in debugging however, in latest version there is a change in the log output. In 2.15 out put is shown below,

Jan 13, 2012 5:42:00 AM org.openqa.selenium.remote.RemoteWebDriver execute
INFO: Executing: [null, newSession {"desiredCapabilities":"Capabilities [{platform=WINDOWS, ensureCleanSession=true, browserName=internet explorer, version=}]"}]
Jan 13, 2012 5:42:16 AM org.apache.http.impl.client.DefaultRequestDirector tryExecute
INFO: I/O exception (java.net.SocketException) caught when processing request: Software caused connection abort: recv failed
Jan 13, 2012 5:42:16 AM org.apache.http.impl.client.DefaultRequestDirector tryExecute
INFO: Retrying request
Jan 13, 2012 5:42:16 AM org.openqa.selenium.remote.RemoteWebDriver execute
INFO: Executing: [e36c6b06-ab81-4daf-b99a-1af8cbc138cd, implicitlyWait {"ms":300000}]
Jan 13, 2012 5:42:17 AM org.openqa.selenium.remote.RemoteWebDriver execute
INFO: Executing: [e36c6b06-ab81-4daf-b99a-1af8cbc138cd, get {"url":"https://test.com/"}]
Jan 13, 2012 5:42:54 AM org.openqa.selenium.remote.RemoteWebDriver execute
INFO: Executing: [e36c6b06-ab81-4daf-b99a-1af8cbc138cd, getCurrentWindowHandle {}]
Jan 13, 2012 5:42:54 AM org.openqa.selenium.remote.RemoteWebDriver execute
INFO: Executing: [e36c6b06-ab81-4daf-b99a-1af8cbc138cd, getCurrentWindowHandle {}]
353b31e2-046e-403f-96da-4f520163d341
Jan 13, 2012 5:42:54 AM org.openqa.selenium.remote.RemoteWebDriver execute
INFO: Executing: [e36c6b06-ab81-4daf-b99a-1af8cbc138cd, switchToFrame {"id":0}]
Jan 13, 2012 5:42:54 AM org.openqa.selenium.remote.RemoteWebDriver execute
INFO: Executing: [e36c6b06-ab81-4daf-b99a-1af8cbc138cd, findElement {"using":"name","value":"a"}]
Jan 13, 2012 5:43:18 AM org.openqa.selenium.remote.RemoteWebDriver execute
INFO: Executing: [e36c6b06-ab81-4daf-b99a-1af8cbc138cd, findElement {"using":"name","value":"b"}]
Jan 13, 2012 5:43:18 AM org.openqa.selenium.remote.RemoteWebDriver execute
INFO: Executing: [e36c6b06-ab81-4daf-b99a-1af8cbc138cd, sendKeysToElement {"id":"e9a6a8d3-301f-49a7-85fc-977f35414095","value":["test"]}]
Jan 13, 2012 5:43:30 AM org.openqa.selenium.remote.RemoteWebDriver execute
INFO: Executing: [e36c6b06-ab81-4daf-b99a-1af8cbc138cd, findElement {"using":"name","value":"p"}]
Jan 13, 2012 5:43:33 AM org.openqa.selenium.remote.RemoteWebDriver execute
INFO: Executing: [e36c6b06-ab81-4daf-b99a-1af8cbc138cd, sendKeysToElement {"id":"c9373957-5fcd-4a63-999f-bf54fa1cc9fd","value":["test"]}]
Jan 13, 2012 5:43:40 AM org.openqa.selenium.remote.RemoteWebDriver execute
INFO: Executing: [e36c6b06-ab81-4daf-b99a-1af8cbc138cd, submitElement {"id":"c9373957-5fcd-4a63-999f-bf54fa1cc9fd"}]
Jan 13, 2012 5:43:59 AM org.openqa.selenium.remote.RemoteWebDriver execute
INFO: Executing: [e36c6b06-ab81-4daf-b99a-1af8cbc138cd, switchToWindow {"name":"353b31e2-046e-403f-96da-4f520163d341"}]

However, in 2.20 the output is like this,

Apr 6, 2012 12:17:02 PM java.util.logging.LogManager$RootLogger log
INFO: Logging of firefox driver is enabled
Apr 6, 2012 12:17:02 PM java.util.logging.LogManager$RootLogger log
INFO: Logging of firefox driver is enabled
Tabsv2.conf
Apr 6, 2012 12:17:02 PM org.openqa.selenium.remote.RemoteWebDriver log
FINE: Executing: get
Apr 6, 2012 12:17:02 PM org.openqa.selenium.remote.RemoteWebDriver log
FINE: Executing: getCurrentWindowHandle
Apr 6, 2012 12:17:04 PM org.openqa.selenium.remote.RemoteWebDriver log
FINE: Executing: getCurrentWindowHandle
{339289c1-afc6-4352-8ba5-b8e5f5c9997c}
Apr 6, 2012 12:17:04 PM org.openqa.selenium.remote.RemoteWebDriver log
FINE: Executing: switchToFrame
Apr 6, 2012 12:17:04 PM org.openqa.selenium.remote.RemoteWebDriver log
FINE: Executing: findElement
Apr 6, 2012 12:17:04 PM org.openqa.selenium.remote.RemoteWebDriver log
FINE: Executing: findElement
Apr 6, 2012 12:17:04 PM org.openqa.selenium.remote.RemoteWebDriver log
FINE: Executing: sendKeysToElement
Apr 6, 2012 12:17:04 PM org.openqa.selenium.remote.RemoteWebDriver log
FINE: Executing: findElement
Apr 6, 2012 12:17:04 PM org.openqa.selenium.remote.RemoteWebDriver log
FINE: Executing: sendKeysToElement
Apr 6, 2012 12:17:05 PM org.openqa.selenium.remote.RemoteWebDriver log
FINE: Executing: submitElement
Apr 6, 2012 12:17:05 PM org.openqa.selenium.remote.RemoteWebDriver log
FINE: Executing: switchToWindow
Apr 6, 2012 12:17:05 PM org.openqa.selenium.remote.RemoteWebDriver log
FINE: Executing: get
Apr 6, 2012 12:17:13 PM org.openqa.selenium.remote.RemoteWebDriver log
FINE: Executing: getCurrentWindowHandle
{339289c1-afc6-4352-8ba5-b8e5f5c9997c}

Notice the difference. Earlier, what the value is passed and what is the text to be typed all printed in the logs. Now, it was changed.

Why? Because they implementation is changed. Earlier this code is used to print the logs in RemoteWebDriver.java

try {
      log(sessionId, command.toString(), command, When.BEFORE);
      response = executor.execute(command);
      if (response == null) {
        log(sessionId, command.toString (), command, When.AFTER);
        return null;
      }

Now, it changed to like this,

try {
      log(sessionId, command.getName(), command, When.BEFORE);
      response = executor.execute(command);
      if (response == null) {
        log(sessionId, command.getName(), command, When.AFTER);
        return null;
      }

So, I changed those lines in source, complied and replaced in the jar file Now, the logs look like as in 2.15.0

Happy Testing.

Locate anything with jQuery in Selenium

One of major issues of automation of XHTML or Ajax based pages is locating the elements which dont have id/name. There is no need for the developers to put them in that way or testers can't go and enforce id/name for all the elements.

First shortcut testers take is XPath locators. Yes, that is easy and can work in any browser, but the pitfall is fragility. A addition of row in that page can make the XPath locator to brake. This issue can be solved by using jQuery's :has and :contains options.

Let see an example,

See the below screenshot,

Code for this html can be found here, http://pastebin.com/6ndmTDST

In that, there is no id,name or any other CSS attribute to locate expect that input and a tag.

What is the locator for locating first checkbox?

jQuery(':has(input):contains("AA"):last input')

Locator for EB in the first row if EB and 1EA both has value of EB like the below image?

jQuery(':has(:contains("EB")):contains("AA"):last :contains("EB"):last')

How it works?


:has locates all the elements which have the element we are looking for. Then, first :contains filters out the last td/tr/div/span in which both elements are located. This takes out the necessity of juggling with parent,child,adjacent calculations. Once we have the row, it is easy to filter the element we are looking for.


You can locate pretty much anything using these two combinations. Next time, if you think some locator can't located without going for XPath, think again. 


Now, comes the next question, WebDriver aka  Selenium 2 doesn't support jQuery. It only support CSS selectors. How can I use these locators in Selenium? Well, you can inject jQuery into the page using JavaScript executor method. Code for the same can be found elsewhere or look at the Selenium/WebDriver users list. 

Note: You'll notice that, there is no jQuery in the HTML code. I used FireQuery.


Happy Automated Testing.

Manual vs Automation : Grinder Analogy

If you're looking for info about Grinder automation framework, you're in wrong place.

After having hard time to explain my colleagues and friends about how not to do automation, I came up with an analogy they can understand better.

It is Grinder. Actually wet grinder, used to make wet flour from rice, from which idilis(Rice cakes) were made. Traditional Indian way of grinding the rice is using attu-kal(grinding stone) which consist of stone with hole and another cylindrical shape which will fit into the hole. Flour is made by rotating the top cylindrical stone.

After electricity, people tried find a way to machine which does this. Of course, you can get the same flour from blade grinder or mixier grinders with out worrying about stone. But that is a different story. So solving this problem is quite difficult unless they figured out machine work shouldn't be an exact replication of manual work. Then the solution is very simple, just rotate the bottom stone. Bingo.

See the images below,

Traditional bottom stone and top cylindrical stone

Modern wet - grinder

we can see this in washing machines, blade grinders etc.Same applies to UI automation.

When you're doing automation, don't start with the manual scripts or how you did that manually. That will not help in making the automation or showing high level of productivity. If automation delivers the same productivity then there no point in doing it.

Just throw away the manual testing procedure, start afresh for automating your product. You'll see a sea change.

Organization and Corruption

Why companies fail? Why companies fail to innovate and becomes a footnote in history?

Those are questions comes to my mind, when I read J Krishnamurthi's saying about corruption because being a part of a organization. His sayings can be summarized into organization and corruption is inseparable. An organization/institutions will corrupt by it own nature.

There are umpteen examples for this. I can't remember a organization or company which is not corrupted. Here corruption = utterly broken. i.e not doing what it intended to do.

I wondered, why this is happening? What is making the these group of people to do nutty things? Is it because they are mobs, Mob rule or herd behavior? All these theories say, there is some magic happen when people act collectively. I don't think, rational man will act irrationally when they form a group. But we see this all the time. So there is something happens when a mob formed.

For me, that something is questioning, lack of it. It can also be said as these institutions lack democracy. Only difference between democracy and mob behavior is questioning. In democracy, people have right to question, in mob rule that is not there. All the authoritative, autocratic, non democratic regimes are corrupt because there is no checks, no questioning.

Read the Monkey-Banana-Ice water story. Take from here, http://www.motivation-for-dreamers.com/never-assume.html. Please the whole article in that link.

Michael Michalko in the second edition of the book Thinkertoys, gives a very interesting example of behaviour that is based on assumptions. He asks the reader to imagine a cage containing five monkeys. A banana is then hung on a string inside the cage and a set of stairs placed in the cage leading up to the banana. Whenever a monkey goes up the stairs and grabs the banana, ice-cold water is poured on all the monkeys in the cage. Very soon, as the monkeys begin to associate touching the banana with being sprayed with ice-cold water they will try to prevent one another from trying to get the banana.
The cold water is then turned off and one monkey removed from the cage and replaced with another that doesn’t know anything about the cold water. This new monkey will inevitably try to get the banana, but the other four monkeys will attack him to stop him from doing so.
If another of the old monkeys is removed and replaced with a newcomer, this newcomer will try to grab the banana. The previous newcomer will gladly take part in beating him up together with the other monkeys in order to prevent him from climbing the stairs to the banana.
Slowly, all the original monkey can be replaced with new monkeys in this manner. The cage will now be totally filled with monkeys that know nothing about the ice-cold water, but they will all not try to get the banana and continue to attack any monkey that tries to do so. “No monkey ever again approaches the stairs. Why Not? Because as far as they know, that’s the way it’s always been around here. Don’t be a monkey. Challenge all assumptions.” These are Michalko’s words of advice.

Is institutions fail because they are not democratic? I think, they are.

Democracy means active participation in decision making. People dont be mute spectators if something goes wrong.  But in organizations? If you disagree with your boss, you quit. Simple solution, isn't it? One may ask, what if boss and I had differences but not to the point of quitting the company. Understanding the difference between election and revolution will add some clarity.

Election gives formal right to say what one thinks about that administration/Govt.It happens in correct intervals. People presented with a choice.

Revolution is some thing like a extreme step of quitting the company. You run out of all the options so you take the extreme step of agitating, protesting most of time violently and bring a change.

In all circumstances, I like the first option. If companies give the right to the employee to say in company matters by some process, then it is hard for the company to ignore the issues. In reality, most of the participation in decision making done by whistle blowing or speaking up. How many people will brave for it? Lot people wont take that road. They simply quit, move to another company and same story repeats endlessly.

There are exceptions to this common phenomena. Companies led by a visionary wont suffer by lack of questioning. But those are rare exceptions.

Now comes the implementation. How to make companies democratic?. I will try to answer that in another post.

WebSite Performance data using Selenium

I wanted to export the Firebug's HTTP Traffic data shown in the Net panel if I crawl the page using Selenium . Why Selenium? There are other alternatives to get that data. However, I have two reasons for this approach.

1. I wanted to know what data from which URL. This will combine the data with URL. i.e. seeing what user will see. It is easier to debug instead of just seeing the requests.
2. Data from Firebug is better then alternatives.

It is easy to see that manually. How to automate that? By using NetExport Add-on by Jan Odvarko. It exports the data as HAR archive.

How to get that working in Selenium WebDriver?

      Create a profile for WebDriver using firefox -P -no-remote.Then, set the AutoExport option to true in Selenium. Below code does that

ProfilesIni allProfiles = new ProfilesIni();
FirefoxProfile profile = allProfiles.getProfile("<ProfileName>");
profile.setPreference("extensions.firebug.netexport.alwaysEnableAutoExport",
driver = new FirefoxDriver(profile);

Now it will export the HTTP traffic data whenever the page is refreshed or new page loaded. Use HARViewer to see the data.

Now comes the tricky part. How to export the data for Ajax requests? I don't know. Netexport exports data, if and when page is reloaded. It can be done, if Selenium can execute the chrome functions. I'm working on that.