I know what I'm trying to do is simple but it's causing me grief. I'd like pull data from HTML using BeautifulSoup. To do that I need to properly use the .find() function. Here's the HTML I'm working with:
<div> <div> <img src="" /> <h4>Ed Boon</h4> <span><a href="" target="_blank">@noobde</a></span> </div> <div> <div></div> <div> <span data-value=73599>73,599</span><span> Real</span><br /> <span data-value=32452>32,452</span><span> Fake</span><br /> <h6>Followers</h6> </div> </div> <div> <img src="//" /> <div> 69% </div> <h6>Audit score</h6>The values I want are 73599 from data-value=73599, 32352 from data-value=32452, and the 69% from percentage good.
Using past code and online examples, this is what I have so far:
RealValue = soup.find("div", {"class":"real number"})['data-value']
FakeValue = soup.find("audit", {"class":"fake number"})['data-value']Both so far to no effect. I'm not sure how to craft the find in order to pull the 69% number.
1 Answer
soup.find("div", {"class":"real number"})['data-value']
Here you are searching for a div element, but the span has the "real number" class in your example HTML data, try instead:
soup.find("span", {"class": "real number", "data-value": True})['data-value']Here we are also checking for presence of data-value attribute.
To find elements having "real number" or "fake number" classes, you can make a CSS selector:
for elm in soup.select(".real.number,.fake.number"): print(elm.get("data-value"))To get the 69% value:
soup.find("div", {"class": "percentage good"}).get_text(strip=True)Or, a CSS selector:
soup.select_one(".percentage.good").get_text(strip=True)
soup.select_one(".score .percentage").get_text(strip=True)Or, locating the h6 element having Audit score text and then getting the preceding sibling:
soup.find("h6", text="Audit score").previous_sibling.get_text(strip=True)
