Extracting text from HTML file using Python
I'd like to extract the text from an HTML file using Python. I want essentially the same output I would get if I copied the text from a browser and pasted it into notepad.
- Filter out HTML tags and resolve entities in python
- Convert XML/HTML Entities into Unicode String in Python
html2text is a Python program that does a pretty good job at this.
NOTE: NTLK no longer supports clean_html function
Original answer below.
I wasted my 4-5 hours fixing the issues with html2text. Luckily i could encounter NLTK. It works magically.
import nltk from urllib import urlopen url = "http://news.bbc.co.uk/2/hi/health/2284783.stm" html = urlopen(url).read() raw = nltk.clean_html(html) print(raw)