I am working on a program that would display a wide range of martial arts. I figured that web scraping would be the best way to do this so that as things grow and evolve, the program would, hopefully, work with the new additions. I had an idea about how to call the wanted unordered list on the page, but the ways I know would require it to have a class/class title, which it doesn't. Any ideas on how to get this list to populate properly in my code? I am fairly new to python, but have some background in other languages
This is the code I found for calling the unordered list:
import requestsfrom bs4 import BeautifulSoupMA_list = "https://en.wikipedia.org/wiki/List_of_Japanese_martial_arts"def MartialArts(): # the target we want to open url = MA_list #open with GET method resp=requests.get(url) #http_respone 200 means OK status if resp.status_code == 200: print("Successfully opened the web page") print("Martial Arts :-\n") soup = BeautifulSoup(resp.text, 'html.parser') # l is the list which contains all the text ie the various arts l = soup.find("ul", {"class" : "class_title"}) for i in l.findAll("a"): print(i.text) else: print("Error") MartialArts()I was expecting the ul to have a class, so this was doomed to fail from the get go. I have a picture below of what I am trying to access. Each of the lists (there are about 30 under the ul block) contains the name of a unique art I am trying to put into a list or loop to display. Wiki page source code div block trying to access and print