PDFMiner

Python PDF parser and analyzer

Last Modified: Sat Jan 10 20:18:36 JST 2009

What's It?

PDFMiner is a suite of programs that aims to help analyzing text data from PDF documents. It includes a PDF parser, a PDF renderer (though only rendering text is supported for now), and a couple of nice tools to extract texts. Unlike other PDF-related tools, it allows to obtain the exact location of texts in a page, as well as other layout information such as font size or font name, which could be useful for analyzing the document.

Features:

Written entirely in Python. (for version 2.5 or newer)
PDF-1.7 specification support.
Non-ASCII languages and vertical writing scripts support.
Various font types (Type1, TrueType, Type3, and CID) support.
Basic encryption (RC4).
PDF to HTML conversion (with a sample converter web app).
Outline (TOC) extraction.
Tagged contents extraction.

Homepage:
http://www.unixuser.org/~euske/python/pdfminer/index.html

Download (source):
http://www.unixuser.org/~euske/python/pdfminer/pdfminer-dist-20090110.tar.gz (1.8Mbytes)

Svn repository:
http://code.google.com/p/pdfminerr/source/browse/trunk/pdfminer

Demo: (online pdf -> html conversion)
http://pdf2html.tabesugi.net:8080/

How to Install

Install Python 2.5 or newer.
Download the PDFMiner source.
Extract it.
Go to the pdfminer directory.

Do the following test:

$ python -m pdflib.pdf2txt samples/simple1.pdf
<html><head><meta http-equiv="Content-Type" content="text/html; charset=ascii">
</head><body>
<div style="position:absolute; top:50px;"><a name="1">Page 1</a></div><span style="position:absolute; border: 1px solid gray; left:0px; top:50px; width:612px; height:792px;"></span>
<span style="position:absolute; writing-mode:lr-tb; left:100px; top:224px; font-size:22px;"> </span>
<span style="position:absolute; writing-mode:lr-tb; left:106px; top:224px; font-size:22px;">Hello </span>
<span style="position:absolute; writing-mode:lr-tb; left:168px; top:224px; font-size:22px;">World </span>
<span style="position:absolute; writing-mode:lr-tb; left:100px; top:124px; font-size:22px;"> </span>
<span style="position:absolute; writing-mode:lr-tb; left:206px; top:124px; font-size:22px;">Hello </span>
<span style="position:absolute; writing-mode:lr-tb; left:368px; top:124px; font-size:22px;">World </span>
<div style="position:absolute; top:0px;">Page: <a href="#1">1</a></div>
</body></html>

Done!

For non-ASCII languages

In order to handle non-ASCII languages (e.g. Japanese), you need to install an additional data called CMap, which is distributed from Adobe.

Here is how:

Get http://www.unixuser.org/~euske/pub/CMap.tar.bz2
Do the follwoing:
```
$ tar jxf CMap.tar.bz2
```
Put the CMap directory into the pdfminer directory.
Go to the pdfminer directory.
Do the follwoing: (this is optional but highly recommended)
```
$ make cdbcmap
```

How to Use

PDFMiner comes with two programs: pdf2txt.py and dumppdf.py.

pdf2txt.py

pdf2txt.py extracts text contents from a PDF file. It extracts all the texts that are to be rendered programatically. It also extracts the corresponding locations, font names, and font sizes for each text portion. However, it cannot extract texts embedded within images (i.e. it does not do optical character recognition). You can provide a password for protected PDF documents whose access is limited.

For non-ASCII languages, you can specify the output encoding (such as UTF-8). Note that not all characters in a PDF can be converted safely to Unicode, as some of them are not included in the current Unicode Standard.

Examples:

$ python -m pdflib.pdf2txt -o output.html samples/naacl06-shinyama.pdf
(extract text as an HTML file whose filename is output.html)

$ python -m pdflib.pdf2txt -c euc-jp samples/jo.pdf
(extract Japanese texts in vertical writing, CMap is required)

$ python -m pdflib.pdf2txt -P mypassword secret.pdf
(extract texts from an encrypted PDF file with a password)

Options:

-o filename

Speficies the output file name. By default, it prints the extracted contents to stdout.

-p pageno[,pageno,...]

Speficies the comma-separated list of the page numbers to be extracted. Page numbers are starting from one. By default, it extracts texts from all the pages.

-c codec

Speficies the output codec for non-ASCII texts.

-t type

Speficies the output format. The following formats are currently supported.

html : HTML format. (Default)
sgml : SGML format.
tag : "Tagged PDF" format. A tagged PDF has its own contents annotated with HTML-like tags. pdf2txt tries to extract its content streams rather than inferring its text locations. Tags used here are defined in the PDF specification (See §10.7 "Tagged PDF").

-P password

Provides the user password to open the PDF file.

-d

Increases the debug level.

dumppdf.py

dumppdf.py dumps the internal contents of a PDF file in pseudo-XML format. This program is primarily for debugging purpose, but it's also possible to extract some meaningful contents (such as images).

Examples:

$ python -m tools.dumppdf -a foo.pdf
(dump all the headers and contents, except stream objects)

$ python -m tools.dumppdf -T foo.pdf
(dump the table of contents)

$ python -m tools.dumppdf -r -i6 foo.pdf > pic.jpeg
(extract a JPEG image)

Options:

-a

Instructs to dump all the objects. By default, it only prints the document trailer (like a header).

-p pageno

Speficies the page number to be extracted. Multiple -p options are allowed. Note that page numbers start from one.

-r (raw)

-b (binary)

-t (text)

Speficies the output format of stream contents. Because the contents of stream objects can be very large, they are omitted when none of the options above is specified.

With -r option, all the stream contents are dumped without decoding. With -b option, the contents are dumped as a binary blob. With -t option, the contents are dumped in a text format, similar to repr() manner. When -r or -b option is given, no stream header is displayed for the ease of saving it to a file.

-P password

Provides the user password to open the PDF file.

-d

Increases the debug level.

Changes

2009/01/10: Handling Type3 font metrics correctly.
2008/12/28: Better handling of word spacing. Thanks to Christian Nentwich.
2008/09/06: A sample pdf2html webapp added.
2008/08/30: ASCII85 encoding filter support.
2008/07/27: Tagged contents extraction support.
2008/07/10: Outline (TOC) extraction support.
2008/06/29: HTML output added. Reorganized the directory structure.
2008/04/29: Bugfix for Win32. Thanks to Chris Clark.
2008/04/27: Basic encryption and LZW decoding support added.
2008/01/07: Several bugfixes. Thanks to Nick Fabry for his contribution.
2007/12/31: Initial release.
2004/12/24: Start writing the code out of boredom...

Related Projects

Terms and Conditions

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Yusuke Shinyama