https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id4

中文版BeautifulSoup库

作用

1. 提取HTML和XML文档中的数据

2. 修改、导航、查找文档

创建html_doc

>>> html_doc = """
...  
...  
...

The Dormouse's story

Once upon a time there were three little sisters; and their names were 
... Elsie, 
... Lacie and 
... Tillie; 
... and they lived at the bottom of a well.

...

#使用bs4库
>>> from bs4 import BeautifulSoup
>>> soup = BeautifulSoup(html_doc)
>>> print soup.prettify()

 
 
 
 
 

  
    The Dormouse's story
  
 

   Once upon a time there were three little sisters; and their names were
  
    Elsie
  
   ,
  
    Lacie
  
   and
  
    Tillie
  
   ; 
and they lived at the bottom of a well.
 

   ...
 

提取所需的字段
>>> soup.title                                                    #提取标题

>>> soup.title.name
'title'

>>> soup.title.string                                            #提取标题的内容
u"The Dormouse's story"

>>> soup.a                                                        #提取字段信息（第一个）
Elsie

>>> soup.p

The Dormouse's story

查找
>>> soup.find_all('a')    
[Elsie, Lacie, Tillie]

>>> for link in soup.find_all('a'):
...     print link.get('href')                        #提取link, href字段
...
http://example.com/elsie
http://example.com/lacie
http://example.com/tillie