Prerequisites: Beautifulsoup
Beautifulsoup is a Python module used for web scraping. In this article, we will discuss how contents of <li> tags can be retrieved from <ul> using Beautifulsoup.
Modules Needed:
- bs4: Beautiful Soup(bs4) is a Python library for pulling data out of HTML and XML files.
- requests: Requests allow you to send HTTP/1.1 requests extremely easily. This module also does not comes built-in with Python.
Approach
- Import the modules
- Provide an URL that has ul and li tags
- Make the requests
- Create the beautifulsoup object
- Find the required tags
- Retrieve the contents under li
Below the code, the HTML snippet contains a body with ul and li tags that have been obtained by the beautifulsoup object.
Method 1: Using descendants and find()
In this method, we use the descendants attribute present in beautifulsoup which basically returns a list iterator object having all the descendants/children of the parent tag, here parent is <ul> tag.
First, import the required modules, then provide the URL and create its requests object that will be parsed by the beautifulsoup object. Now with the help of find() function in beautifulsoup we will find the <body> and its corresponding <ul> tags. After this, the descendants attribute will give us the list iterator object which is needed to convert back into list. This list has a next line item, the tags with text, and finally the only text. So, we will print every second successive element of the list.
Example:
Python3
# importing the modulesimport requestsfrom bs4 import BeautifulSoup # providing url # creating requests objecthtml = requests.get(url).content # creating soup objectdata = BeautifulSoup(html, 'html.parser') # finding parent <ul> tagparent = data.find("body").find("ul") # finding all <li> tagstext = list(parent.descendants) # printing the content in <li> tagprint(text)for i in range(2, len(text), 2): print(text[i], end=" ") |
Output:
Method 2: Using find_all()
Approach is same as the above example, but instead of finding the body we will find ul tags and then find all the li tags with the help of find_all() function which takes the tag name as an argument and returns all the li tags. After this we will simply iterate over all the <li> tags and with the help of text attribute we will print the text present in the <li> tag.
Example:
Python3
# importing the modulesimport requestsfrom bs4 import BeautifulSoup # providing url # creating request objectreq = requests.get(url) # creating soup objectdata = BeautifulSoup(req.text, 'html') # finding all li tags in ul and printing the text within itdata1 = data.find('ul')for li in data1.find_all("li"): print(li.text, end=" ") |
Output:
