Make Selenium Wait in Python

Nowadays, the majority of online applications use the AJAX framework. The elements in AJAX pages may load at different time intervals. This means that selenium can attempt to locate an element that is not yet available, leading to an error.

We can fix this problem by implementing waits. These delays give the site elements time to load fully. In this article, we will discuss three kinds of waits (with examples):

  • Hard wait – using time.sleep(),
  • Implicit wait, and,
  • Explicit wait.

At the end of the post, you should have a solid understanding of these waits and when to use them.

Hard wait – using time.sleep()

Python time.sleep(<secs>) function can be used to halt the execution of a Python script for a given number of seconds. With this wait, we hope the website we are trying to access with selenium will finish loading.

In the following example, we use time.sleep() to halt execution for 10 seconds, locate web element with ID=”Culture”, stop the program execution for another 10 seconds, then attempt to locate another element with ID=”Element22″.

Output:

It took 10.048056840896606 seconds.
Element count not be found. Timed out.
It took 10.01723027229309 seconds.

Unlike the first element, the second element does not exist on the web page (NoSuchElementException was raised and handled using a try-except block); therefore, waiting will not help us here.

Assuming that the first element became available after 2 seconds of loading the site, time.sleep(10) will still wait for 10 seconds. That means 8 seconds of unnecessary wait. Is there something to do about this? Yes, by using explicit or implicit waits. Let’s discuss them next.

Implicit wait

When searching for any element (or elements) that aren’t immediately available, WebDriver is instructed via an implicit delay to poll the DOM for a pre-determined period.

The implicit wait is set once and remains set for as long as the WebDriver object remains alive.

By default, selenium does not implement implicit wait.

Unlike time.sleep(), implicit wait proceeds with program execution as soon as the sought element is found. For example,

driver.implicit_wait(10) polls the DOM for 10 seconds when finding any element below it on the script. If a given element is located within 10 seconds, execution continues; otherwise, NoSuchElementException is raised.

Output:

Available element (Culture):  0.04906463623046875
Element not found.
Element not present (Cultur):  20.023759365081787
Another present element:  0.0120849609375

driver.implicitly_wait(20) implemented 20 seconds wait for each element after it was defined, and the execution is resumed as soon as the element is available.

If the element is not found after the pre-determined time (like for Cultur), NoSuchElementException is raised.

Explicit Wait

An explicit wait is a code you define for a specific condition to occur before continuing the code execution.

In most cases, this involves using two functions:

  • selenium.webdriver.support.ui.WebDriverWait() to implement the wait on the Web Driver object, and,
  • Selenium.webdriver.support.expected_conditions() to set the conditions.

WebDriverWait(driver, timeout, poll_frequency= 0.5).until(method) – where the expected_conditions are defined on the until method.

By default, WebDriverWait calls the until method every half a second (500 milliseconds) until it returns success. The expected_conditons will return True in case of success or not null if it fails to locate an element.

If expected_conditions fails to locate the specified element in the allocated time, it throws a TimeoutException.

Output:

Element count not be found. Timed out.
It took 20.400476694107056 seconds.
It took 0.4250028133392334 seconds.

WebDriverWait was timed out in the first case because the element didn’t exist. After that, TimeoutException was raised.

In the second case, the WebDriverWait can wait for 20 seconds, but the element was found sooner (0.42 seconds), reducing the waiting time.

Expected Conditions

In the example above, we used the presence_of_element_located() function to define the expected conditions. Other common expected conditions include:

  • element_to_be_clickable(mark),
  • element_to_be_selected(element),
  • staleness_of(element),
  • title_contains(title), and,
  • text_to_be_present_in_element(locator, text)

You can see the complete list on the documentation of webdriver.support.expected_conditions.

Conclusion

We have discussed three kinds of waits in this article. As shown, using time.sleep() is not a good practice in most cases because it halts the execution of the code for a specified time without checking if the page is loaded.

To avoid this, use explicit or implicit waits. The implicit wait can be defined on top of the script (after initializing the WebDriver object) so that the pre-determined delay can be implemented for every element in the code. Alternatively, you can define an explicit wait for the specific elements of your choice.