Python3 program to print all li (HTML List items element) tag values for a supplied website

gistfile1.txt

# Python3 program to print all li tag values for a supplied website
import requests
import argparse
from bs4 import BeautifulSoup


print(help)
parser = argparse.ArgumentParser(description='This is a program to print all li tags')

parser.add_argument("-url", help="enter the URL you want to parse. eg. https://www.mirraw.com/")
args=parser.parse_args()

if args.url:
    url = args.url
else:
    url = input('Enter the URL for which you want all li tag value printed: ')


def get_html (url):
  reqs = requests.get(url)
  return reqs
  

def find_all_li(reqs):
  soup = BeautifulSoup(reqs.text, 'lxml')
  print("\nFind and print all li tags:\n")
  for tag in soup.find_all("li"):
    #print("{0}: {1}".format(tag.name, tag.text))
    print("{1}".format(tag.name, tag.text))


find_all_li(get_html(url))

How to run this program

Steps

1. Make sure python 3 is installed (independent of the operating system)
   You may setup a virtual environment if you want other wise just install this package as mentioned blow How to setup Virtual Environment
2. Assuming it's installed as python/pip run pip install beautifulsoup4 to install Beautiful soup package
3. To run python <filename.py> -url https://example.com
   Alternatively if you don't supply the url in the flag it will prompt you for an input

ented Dec 12, 2022 •

You might have to install pip3 install lxml package