Selenium in Depth – the demos

March 6, 2017

6 comments

Last month I gave a talk on the Israel Test Automation Meetup titled “Selenium in Depth”. In this talk I showed highly simplified examples that demonstrate how Selenium behaves in certain conditions, mostly related to the html and JavaScript on the page.

Now I decided to share the demos and you can find it here.

Each demo is contained in a Test Case method. Most of them use a specific super-simple HTML file that is part of the project too. Note that not all tests pass, and this is by design!

Here’s a short description of the test cases:

  1. IsItABugInSelenium – If you just open the file IsItABugInSelenium.html in the browser, it loads fine. If you run the test, you’ll get a general WebDriverException error that many people think of as an internal Selenium problem. However, if you’ll try to click the button on the page manually, you’ll see that the entire browser tab hangs, regardless of Selenium. Open the html file in a text editor and see why…
  2. DoWeReallyNeedWait – Open the file SlowButton.html in the browser and click on the button. You’ll see that the input box is updated only 3 whole seconds afterward. Note that the test does not use any sort of waiting (no Thread.Sleep, no WebDriverWait, no ImplicitlyWait, etc.) between the press of the button and the verification of value in the input box, but if you run the test, you’d probably be surprised to see that it passes! Again, look inside the html file to see what it does.
  3. WhatsTheDifference – If you open the file SlowButton.1.html in the browser, you’d see that it behaves very similarly to the page in the previous example. However, this test (which is identical to the previous one, except of the html file), fails. The difference is that this file uses the JavaScript function window.setTimeout which sets a timer to the specified period and executes the callback when it’s due, rather than doing a tight-loop like the previous example. This has the effect of freeing the main thread which also causes the Click method to return before the callback that updates the value of the input element is invoked. Note that this behavior is very similar to what happens on an AJAX call. This is a more common scenario that require the test to wait.
  4. WillThisHelp – This example is identical to the previous one, except of that line that was added at the beginning of the test:
    _driver.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(10));
    This should solve the problem, isn’t it? Well, if you run the test you’ll see that it doesn’t. The reason that it doesn’t is that ImplicitlyWait only affects finding elements (i.e. the methods IWebDriver.FindElement and IWebDriver.FindElements), but the test fails because GetAttribute(“value”) doesn’t wait for the value to change. If you think about it for a moment, it makes sense because Selenium can’t know when the value is actually ready…
  5. ExplicitWait – This example shows how to actually solve the problem of the previous test by using an explicit wait (using the WebDriverWait class).
  6. WaitAndIgnoreExceptions – This example shows two things regarding WebDriverWait. The first is that it can be used to perform any operation regardless of Selenium. In this example we wait for a file to be created. The second this is the IgnoreExceptionTypes method. To see this test in action, before running the test, open Notepad, type “Hello” and save it as C:\temp\test.txt, but leave Notepad open. Then run the test and notice that it’s running doing nothing. Only when you click Save in Notepad the test completes. Explanation: first the test deletes the file to ensure that it doesn’t exist; then it uses the WebDriverWait to read the content of the file. While the file does not exist, the lambda passed to the WebDriverWait is called but throws FileNotFoundException. But because we specified that type of exception in the IgnoreExceptionTypesWebDriverWait continues to invoke the lambda continuously until you re-create the file through Notepad. WebDriverWait keeps invoking the delegate that you pass to it until it doesn’t throw an exception and returns something that is not null or false.
  7. ObviousNoSuchElementException – This example starts the discussion on the various Selenium exceptions, specifically NoSuchElementException and StaleElementReferenceException. It shows that obvious fact that if an element does not exist, a NoSuchElementException is thrown. Pay attention that this exception is thrown from the FindElement method and not from the Click method (this is still quite obvious…)
  8. WhenImplicitlyWaitHelps – In this example we finally see when ImplicitlyWait really helps (unlike in the WillThisHelp test). The point to notice here that clicking the button sets a timer that creates a new element when it’s due, and that we don’t have any explicit wait before we use FindElement to find the new element. You can try to comment out the first line, which uses the ImplicitlyWait and see that without it the test fails.
  9. StaleReferenceElementDemo – Many automation developers hate, or afraid from this exception, because they don’t fully understand what it means. Here’s a very simple example that cause it: the button in RemoveElement.html removes the input element from the DOM. In the test, we find the input element before we click the button, so FindElement succeeds. However, after clicking the button, the input element disappears and therefore any method or property that we invoke on the input element will consequently fail with StaleElementReferenceException.
    One important thing to note: StaleElementReferenceException is never thrown from invoking methods on the IWebDriver (e.g. IWebDriver.FindElement), only from methods on IWebElement (though IWebElement.FindElement may throw it).
    Other cases that this exception can occur is when you try to use an element that you found on one page after you already navigated to another page; or when you found an element on one iframe or window, but used SwitchTo to switch to another iframe or window. This is because an iframe is essentially another page.
  10. UnjustifiedStaleElement – here’s another example of when StaleElementReferenceException might occur, even though it’s less obvious. If you open the file SeemlessStaleElement.html in the browser and open the developer tools (F12), you’ll see that the DOM remains exactly the same when you click the button (though you may notice that the Input element blinks when click the button). So how come this test fails on StaleElementReferenceException? If you’ll look at the JavaScript code on the page, you’ll see that when you click the button, it essentially removes and re-creates the input element (this is why you see it blink in the Developer tools). Even though that the element is identical before and after the click, it’s actually not the same one! This is why we get the StaleElementReferenceException. Note that some JavaScript frameworks do such things implicitly, and this can be very confusing. In this case you’ll probably won’t be able to avoid using try/catch to handling this case and retry Finding the element first and then repeat the failed operation. But my advise is that you first make sure that this is indeed the case you see, and if you do, catch the StaleElementReferenceException specifically and in the smallest try block. Otherwise you’d may hide other problems that you didn’t intend to.
  11. WhatHappensWhenFindElementsFindsNothing – Before running this test, try to answer this question: what happens when FindElements (note the plural form!) finds no matching element?
    1. Throws NoSuchElementException
    2. Returns null
    3. Returns an empty list (IReadOnlyCollection<IWebElement>)

    If you’d run the test, you’ll see that the correct answer is 3

  12. FindElementsAndExplicitlyWait – Now, what do you think would happen if you’re using FindElements after you used ImplicitlyWait? There are 2 cases to consider: the first is what happens if there are some matching elements when the method is called?
    1. It returns immediately with the elements that it found
    2. It waits the duration specified in the ImplicitlyWait anyway, in order to allow for additional elements to be added during this period (e.g. if a list is loaded asynchronously)

    The second case is when there are no matching elements when the method is called?

    1. The method will immediately throw NoSuchElementException
    2. The method will immediately return an empty list
    3. The method will wait until at least one matching element is found and then return (even if the specified during did not elapsed). If it doesn’t happen, it will wait for the entire duration and then will throw NoSuchElementException when the duration elapses.
    4. The method will wait until at least one matching element is found and then return (even if the specified during did not elapsed). If it doesn’t happen, it will wait for the entire duration and then return an empty list.
    5. The method will anyway wait for the specified duration and only then return with whatever elements it found, or an empty list if no element was found.

    The answers to this tricky questions are 1 and 4 (even though in my opinion it should have been 1 and 2, because if the page loads the elements in a list asynchronously, then finding one element doesn’t mean that the entire list was loaded. Instead, I’d prefer to use an explicit wait with some external indication that will tell me if the list was completely loaded or not. But they won’t change it now even if they’ll think I’m right, as it will break backward compatibility…)

  13. WasTheButtonReallyClicked – To understand what’s going on with this demo, open the page Calculate.html in the browser, and also look at its source. As you can see, after you type two numbers in the X and Y fields and press Tab to move the focus away from the Y field, it invokes the displayExpression function which displays the corresponding expression. Only when click the “Calculate” button, it should also write the result. But if you try this out you’ll see that apparently clicking the “Calculate” button does nothing. This is true also when you run the test using Selenium.
    The reason is that I (intentionally) put a typo in the string calcualationResult in line 22 of Calculate.html, which causes an exception in JavaScript. But even though there’s an exception in JavaScript, Selenium doesn’t reflect it to us and the Click method returns successfully. If this bug is easily reproducible like in this case, then it’s easy to diagnose, but sometimes buggy JavaScript code may throw exception only occasionally. In these cases, people (including myself in the past) often blame Selenium for occasionally missing to actually perform the click operation. But the truth is that in all cases (at least those I encountered and investigated after I knew this little fact), the blame is not in Selenium, nor in your test code, but rather it’s a bug in the web page’s JavaScript!

The other C# source file, InvestigateFailures.cs, continues the narrative of the failing Calculate page, and shows some techniques that help investigate failures:

  1. TakeScreenshot – shows how to take a screenshot
  2. TakeDocumentSource – shows how to save the HTML source. Note that different browsers behave slightly different in this method, but at least Chrome returns an HTML that reflects the current state of the DOM (and the actual page source). This is very useful to investigate NoSuchElementException and StaleElementReferenceException as you can later open the file in Notepad (or even the browser!) and see if the element exists or not. Note that if you open the file in the browser, in some cases it will show correctly, but in other cases it won’t, as it depends on additional files (css, javascript, etc.) in order to render correctly. But even if it doesn’t appear correctly, you can still locate the elements in most cases.
  3. LogEvents – this shows the use of the EventFiringWebDriver to automatically write every click to the log. You can of course use this class to write all other supported events to the log too. Unfortunately, some events, e.g. ElementValueChanging does not provide all the important information like the previous and new values. In fact, I submitted a pull request to fix this, but unfortunately I didn’t get any response yet…
  4. ReadBrowserLogs – this example shows how to use the IWebDriver.Manage().Logs property to retrieve the browser’s log. This is really where JavaScript exceptions are written to. So if you read this log and add it to your test results, then you’ll be able to show the developer exactly what was the problem in case of a JavaScript exception!

The last test class, ExecuteJavaScript.cs shows the ExecuteJavaScript extension method and some interesting usages of it, including returning a value from JavaScript, passing arguments to it, including arguments of type IWebElement, and performing asynchronous JavaScript operations. Note that in the last case, ExecuteAsyncScript allows you execute an asynchronous JavaScript operation but it always waits until it finishes, therefore the method itself is synchronous! This is a bit confusing indeed. If you really want to start some asynchronous JavaScript operation, and only later wait for its result, you should use the regular ExecuteJavaScript method and built it in such a ways that will raise some flag when it completes, then later in the test code, call ExecuteJavaScript again inside a WebDriverWait (or some other loop), to check whether the flag was raised or not.

Let me know if you’ve learned a think or two from this article, or if you have any other comment!

 

Add comment
facebook linkedin twitter email

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

*

6 comments

  1. ReutJuly 5, 2017 ב 9:44

    Hi Arnon, great post!
    Is the use in POM necessarily the use with the static class PageFactory? What is more common- to use with FindsBy attribute (and PageFactory) OR to find the elements with wait.Until? Also, with the use of FindsBy how can we control the timeout, is it the default timeout of FindElement?
    Last question is about the design of my project: where should I locate all the assertions in the WebPage- in the WebPage itself or in separate (enormous) class?

    Reply
    1. Arnon AxelrodJuly 12, 2017 ב 9:17

      Hi Reut!

      Thanks for your questions. I’ll answer them one by one:

      1. Is the use in POM necessarily the use with the static class PageFactory?
      POM (which is an abbreviation for Page Object Model) is primarily a design pattern. That means that it’s not directly related to any specific technology (i.e. Selenium), and that you can implement it in different ways. The concept of a POM is that each page, and also logical areas within a page, should have their own class, which encapsulates the technical details of how to locate and interact with the elements (that is, in private members), and exposes (using public methods), the operations that the user can do with them.
      the PageFactory class helps implement this pattern in simple way using attributes (in .Net) /annotations (in Java).
      Personally, I usually prefer to implement the pattern without the PageFactory class, maybe out of habit, but also I found it somewhat limiting. In particular, I often need an element only in a single (public or private) method, so I prefer to define it as a local variable and not as a field.

      2. What is more common- to use with FindsBy attribute (and PageFactory) OR to find the elements with wait.Until?
      I don’t think that the question “what is more common” is relevant. You should do what works best for you! As I mentioned above, I personally don’t tend to use the PageFactory class, but I know many that do. Note also that IWebDriver.FindElement does not necessarily means that you have to use Wait.Until – as I describe in this post, in many cases it’s completely unnecessary.

      3. Also, with the use of FindsBy how can we control the timeout, is it the default timeout of FindElement?
      First, as described in the post, FindElement does not has a default timeout by itself, as it behaves synchronously. The same goes for FindsBy. I haven’t tried it, by I believe that if you’re using ImplicitWait then it will affect the FindsBy decorated members as well.
      If you don’t want to use ImplicitWait (as it’s too global…), then you can use one of the overloads of PageFactory.InitMembers that accept an IElementLocator argument, and pass to it an instance of RetryingElementLocator. In the constructor of this class you can specify the timeout and also the polling interval.

      4. Last question is about the design of my project: where should I locate all the assertions in the WebPage- in the WebPage itself or in separate (enormous) class?
      In my opinion, no class should be enormous, and therefore putting all the assertions in one big class it’s not a good idea. Some people put assertion methods in the page object themselves, but in my opinion this is not a very good idea either.
      In my point of view, the responsibility for validating the expected results is the responsibility of the test methods themselves, and not the responsibility of the page objects. Moreover, validations (asserts) should normally be very simple (usually compare an actual value to some expected one), and should not contain sophisticated logic. If this is the case, then usually each test has different validations (asserts), and therefore there’s no point in re-using the assertions.
      What I do put in the page objects, are getters for values that the user sees. I then use these getters in the asserts (inside the test method) to compare their values with the expected ones.

      I suggest that you’ll take a look at my previous post: “Writing maintainable automated test and infrastructure – With live video recording!” and watch the video, to learn more about the best practices that I promote.

      Please reply back if you have further questions!

      Reply
      1. ReutJuly 12, 2017 ב 9:37

        Thanks, Arnon your answer is very clear! 🙂
        I also think the assertions are the responsibility of the test methods but if I have to repeat the same assertions in few tests?

        Reply
        1. Arnon AxelrodJuly 13, 2017 ב 9:35

          There’s no one right answer to that… I’d probably prefer to put shared assertions in the test class or a base test class, but you may put them in the page objects or other helper classes too if it makes more sense to you.
          But the first question that must be asked is whether these assertions really have to be verified in multiple tests? Usually it indicates one of the following issues:
          1. Multiple tests are essentially the same, and may be refactored to a data-driven, or parameterized test.
          2. Each test verify too many things, and does not have a clear purpose.
          3. You have code duplication: after a certain operation, you always want to verify that it completed successfully so you end up with multiple places where you call a method that performs the operation, followed by a call to the assertion. In this case, the assertion should be done inside the method that performs the operation in the first place. Note that a general programming best practice is the a method call either succeeds or it throws an exception. I.e., you should not call another method to verify that the previous one succeeds. In cases where a method only begins an operation and does not wait for it to complete and therefore can’t know whether it succeeded or not, the name of the method should reflect it, e.g. BeginDoSomething(). In this cases the method only promises that it should succeed beginning the operation, and not that the operation as a whole completed successfully.

          HTH,
          Arnon.

          Reply
  2. LirJuly 17, 2017 ב 10:40

    Hi, like your post . I have a class that my page objects use for finding the elements, let’s call it ElementFinder. Should I locate that class as a member of each Page Object or should it be a static class and each page object will use its methods for finding its element on the page? i.e. ElementFinder.FindByName(“myName”). What is the best design? Another thought is to call that class PageBase and eacg page object will inherit from it. Please help me to find the best design. Thanks.

    Reply
    1. Arnon AxelrodAugust 10, 2017 ב 17:51

      The question is why do you want to wrap the native FindElement of Selenium and not use it directly? Beware that your wrapper not limit the options, especially to find an element within another element (and not just in the entire page).
      If you do have a good reason to wrap it, here are my guidelines for where to put it:
      1. If this method depends on state of sort (i.e. some field or property), then it shouldn’t be static. Otherwise you should probably make it static. If I’m allowed to guess, then I bet that this method uses an IWebDriver field, so it shouldn’t be made static (also the IWebDriver field should not be static…)
      2. Avoid duplication. If you need the same methods in many classes, then it should probably be either in a base class or in another class that the classes that use it accept in their constructors. If you need *similar* methods, but not identical ones, you should extract the common code to a shared place and keep the differences where they belong. For example, if some page objects represent the entire page and some represent only portions of it, then probably you’d want the class that represents the entire page to have such a FindByName method that operates on the IWebDriver, while the classes that represent portions of the page need a similar method but that operates on the IWebElement of that portion. In that case, you can create one method that uses an ISearchContext field, and put it in the base class. Because both IWebDriver and IWebElement derive from ISearchContext, it will work for both.

      BTW, have you took a look at my TestAutomationEssentials.Selenium project? It also have a wrapper for FindElement (called WaitForElement) among some other good stuff :-). You can find its sources in GitHub (https://github.com/arnonax/TestEssentials) or simply search for the TestAutomationEssentials NuGet package from within Visual Studio.

      HTH,
      Arnon.

      Reply