XPath with contains and starts-with in Selenium Python - Deep Dive

Overview - XPath with contains and starts-with

What is it?

XPath is a way to find elements on a webpage by describing their location or attributes. The functions contains() and starts-with() help find elements whose attributes or text include or begin with certain words. This makes locating elements flexible when exact matches are hard or impossible. These functions are used in Selenium tests to interact with dynamic or partially known page content.

Why it matters

Webpages often change or have dynamic content, so exact matches for element locators can fail. Using contains() and starts-with() lets testers find elements even if only part of the attribute or text is known. Without these, tests would break often, making automation unreliable and frustrating. This flexibility saves time and makes tests more stable.

Where it fits

Before learning this, you should understand basic XPath syntax and how Selenium locates elements. After mastering contains() and starts-with(), you can learn other XPath functions and advanced selectors for more precise or complex element targeting.

Mental Model

Core Idea

contains() and starts-with() let you find elements by matching parts of their attributes or text, not just exact values.

Think of it like...

It's like searching for a book in a library by remembering only part of its title or the first few words, instead of the full exact title.

XPath Expression Structure
┌───────────────────────────────┐
│ //tagname[@attribute_function]│
│                               │
│ contains(@attr, 'value')       │
│ starts-with(@attr, 'value')   │
└───────────────────────────────┘

Build-Up - 7 Steps

1

Foundation

Basic XPath Syntax Refresher

Concept: Understand how XPath locates elements using tags and attributes.

XPath uses paths like //tagname[@attribute='value'] to find elements. For example, //input[@id='username'] finds an input element with id 'username'. This is an exact match on the attribute.

Result

You can find elements by exact attribute matches.

Knowing basic XPath syntax is essential before using functions like contains() or starts-with().

2

Foundation

Why Exact Matches Fail Often

Concept: Learn why exact attribute matches are not always reliable in tests.

Webpages often change attribute values slightly or add dynamic parts like timestamps or session IDs. For example, id='user_1234' changes each time. Exact matches like //input[@id='user_1234'] will fail when the number changes.

Result

Exact XPath locators break easily on dynamic pages.

Understanding this problem motivates the need for partial matching functions.

3

Intermediate

Using contains() for Partial Matches

🤔Before reading on: do you think contains() matches only the start, only the end, or any part of the attribute? Commit to your answer.

Concept: contains() checks if an attribute or text includes a given substring anywhere inside it.

Example: //input[contains(@id, 'user')] finds any input whose id contains 'user' anywhere, like 'user_1234' or 'my_user_id'. This is useful when only part of the attribute is stable.

Result

XPath finds elements with attributes containing the substring.

Knowing contains() lets you write flexible locators that survive minor attribute changes.

4

Intermediate

Using starts-with() for Prefix Matches

🤔Before reading on: does starts-with() match anywhere in the string or only at the beginning? Commit to your answer.

Concept: starts-with() checks if an attribute or text begins with a specific substring.

Example: //div[starts-with(@class, 'alert-')] finds divs whose class starts with 'alert-', like 'alert-error' or 'alert-success'. This is useful when the start of the attribute is stable but the rest varies.

Result

XPath finds elements with attributes starting with the substring.

Using starts-with() helps target elements with predictable prefixes, improving locator precision.

5

Intermediate

Combining contains() and starts-with()

🤔Before reading on: can you combine contains() and starts-with() in one XPath? Predict how that looks.

Concept: You can combine these functions with logical operators to create complex locators.

Example: //a[contains(@href, 'login') and starts-with(@class, 'btn-')] finds links with 'login' in href and class starting with 'btn-'. This narrows down elements precisely.

Result

XPath locates elements matching multiple partial conditions.

Combining functions allows crafting robust locators for complex page structures.

6

Advanced

Using contains() and starts-with() on Text Nodes

🤔Before reading on: do you think contains() and starts-with() work only on attributes or also on visible text? Commit your guess.

Concept: These functions can also match visible text inside elements, not just attributes.

Example: //button[contains(text(), 'Submit')] finds buttons whose visible text includes 'Submit'. Similarly, //h1[starts-with(text(), 'Welcome')] finds headers starting with 'Welcome'. This helps when attributes are missing or unreliable.

Result

XPath finds elements by partial visible text matches.

Matching visible text expands locator options beyond attributes, useful for UI testing.

7

Expert

Performance and Reliability Considerations

🤔Before reading on: do you think using contains() and starts-with() always improves test speed and reliability? Commit your answer.

Concept: While flexible, these functions can slow down XPath evaluation and sometimes match unintended elements if used carelessly.

Using contains() on large attribute values or many elements can slow tests. Overly broad contains() may match multiple elements, causing flaky tests. Experts balance flexibility with specificity, sometimes combining with other conditions or using CSS selectors when possible.

Result

Tests become more stable but may need tuning for speed and precision.

Understanding tradeoffs helps write maintainable, fast, and reliable locators in real projects.

Under the Hood

XPath expressions are parsed and evaluated by the browser or Selenium's engine. Functions like contains() and starts-with() perform substring checks on attribute or text node strings during evaluation. The engine traverses the DOM tree, applying these functions to each candidate element until matches are found.

Why designed this way?

XPath was designed to be a powerful query language for XML and HTML documents. contains() and starts-with() were included to allow flexible matching beyond exact values, addressing the need to handle dynamic or partial data. Alternatives like regex were not included to keep XPath simpler and faster.

DOM Tree Traversal with XPath
┌─────────────┐
│ Root Node  │
└─────┬───────┘
      │
┌─────▼───────┐
│ Candidate   │
│ Elements    │
└─────┬───────┘
      │ Apply contains()/starts-with()
┌─────▼───────┐
│ Filtered    │
│ Matches     │
└─────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does contains(@attr, 'val') only match attributes starting with 'val'? Commit yes or no.

Common Belief: contains() only matches if the attribute starts with the given substring.

Reality: contains() matches if the substring appears anywhere inside the attribute, not just at the start.

Why it matters: Misunderstanding this leads to wrong locators that miss elements or match too few, causing test failures.

Quick: Can starts-with() be used to match text inside elements? Commit yes or no.

Common Belief: starts-with() only works on attributes, not on visible text.

Reality: starts-with() can be used on text() nodes to match visible text starting with a substring.

Why it matters: Ignoring this limits locator strategies, especially when attributes are missing or unreliable.

Quick: Does using contains() always make tests faster? Commit yes or no.

Common Belief: Using contains() always improves test speed because it is flexible.

Reality: contains() can slow down XPath evaluation if used on many elements or large strings.

Why it matters: Blindly using contains() can cause slow or flaky tests, frustrating developers and testers.

Quick: Can combining contains() and starts-with() cause ambiguous matches? Commit yes or no.

Common Belief: Combining these functions always narrows down to a single element.

Reality: If not carefully crafted, combined conditions can still match multiple elements, causing flaky tests.

Why it matters: Assuming combined functions guarantee uniqueness leads to unreliable tests and debugging headaches.

Expert Zone

1

contains() is case-sensitive in XPath 1.0, so matching text with different cases requires workarounds or XPath 2.0 features.

2

Using starts-with() on long attribute values is faster than contains() because it only checks the beginning, reducing evaluation time.

3

Combining contains() with position() or indexing can help select the exact element when multiple matches occur.

When NOT to use

Avoid contains() and starts-with() when exact attribute values are stable and known, as exact matches are faster and more reliable. For complex text matching, consider CSS selectors or JavaScript-based locators. When performance is critical, minimize use of these functions or cache elements.

Production Patterns

In real projects, testers use contains() to handle dynamic IDs or classes, starts-with() for common prefixes like 'btn-' or 'nav-', and combine them with other XPath axes like parent or sibling to pinpoint elements. They also write reusable locator functions in test frameworks to abstract these patterns.

Connections

Regular Expressions

Both are used for pattern matching in strings.

Understanding XPath functions helps appreciate how pattern matching works in other domains like regex, which offers more power but also complexity.

SQL LIKE Operator

contains() and starts-with() are similar to SQL's LIKE '%value%' and LIKE 'value%'.

Knowing this analogy helps testers who know databases understand XPath string matching intuitively.

Human Language Search

Partial matching in XPath resembles how search engines find documents by keywords or prefixes.

Recognizing this connection shows how partial matching is a universal concept in information retrieval.

Common Pitfalls

#1 Using contains() with too generic substring causing multiple matches.

Wrong approach:

//div[contains(@class, 'btn')]

Correct approach:

//div[contains(@class, 'btn-primary')]

Root cause:

Choosing a substring that is too common leads to ambiguous matches and flaky tests.

#2 Using starts-with() on an attribute that does not have a stable prefix.

Wrong approach:

//input[starts-with(@id, 'user_')]

Correct approach:

//input[contains(@id, 'user_')]

Root cause:

Assuming a stable prefix when the attribute varies unpredictably causes missed elements.

#3 Applying contains() or starts-with() on elements without the attribute, causing no matches.

Wrong approach:

//span[contains(@title, 'info')]

Correct approach:

//span[@title and contains(@title, 'info')]

Root cause:

Not checking attribute existence leads to XPath errors or no results.

Key Takeaways

XPath functions contains() and starts-with() enable flexible element location by matching parts of attributes or text.

They solve the problem of dynamic or partially known page content that breaks exact match locators.

Using these functions wisely improves test stability but requires care to avoid slow or ambiguous matches.

Combining these functions with other XPath features allows precise and maintainable locators in real-world testing.

Understanding their behavior and limitations is key to writing robust Selenium tests.