PDFMiner

Python PDF parser and analyzer

Homepage   Recent Changes

Last Modified: Wed Apr 7 10:20:18 UTC 2010

What's It?

PDFMiner is a suite of programs that help extracting some information from PDF documents. Unlike other PDF-related tools, it focuses entirely on getting and analyzing text data. PDFMiner allows to obtain the exact location of texts in a page, as well as other information such as fonts or lines. It includes a PDF converter that can transform PDF files into other text formats (such as HTML). It has an extensible PDF parser that can be used for other purposes instead of text analysis.

Features:

On the performance, PDFMiner is about 20 times slower than other C/C++-based software such as XPdf.

Download from PyPI:
http://pypi.python.org/pypi/pdfminer/

Discussion: (for questions and comments, post here)
http://groups.google.com/group/pdfminer-users/

View the source:
http://code.google.com/p/pdfminerr/source/browse/trunk/pdfminer

Online Demo: (pdf -> html conversion webapp)
http://pdf2html.tabesugi.net:8080/

Install

  1. Install Python 2.4 or newer. (Python 3 is not supported.)
  2. Download the PDFMiner source.
  3. Unpack it.
  4. Run setup.py to install:
    # python setup.py install
  5. Do the following test:
    $ pdf2txt.py samples/simple1.pdf
Hello

World

Hello

World

H e l l o

W o r l d

H e l l o

W o r l d
  6. Done!

For CJK languages

In order to process CJK languages, you need an additional step to take during installation: 
# make cmap
python tools/conv_cmap.py pdfminer/cmap Adobe-CNS1 cmaprsrc/cid2code_Adobe_CNS1.txt cp950 big5
reading 'cmaprsrc/cid2code_Adobe_CNS1.txt'...
writing 'CNS1_H.py'...
...
(this may take several minutes)

# python setup.py install

How to Use

PDFMiner comes with two handy tools: pdf2txt.py and dumppdf.py.

pdf2txt.py

pdf2txt.py extracts text contents from a PDF file. It extracts all the texts that are to be rendered programmatically, ie. text represented as ASCII or Unicode strings. It cannot recognize texts drawn as images that would require optical character recognition. It also extracts the corresponding locations, font names, font sizes, writing direction (horizontal or vertical) for each text portion. You need to provide a password for protected PDF documents when its access is restricted. You cannot extract any text from a PDF document which does not have extraction permission.

Note: Not all characters in a PDF can be safely converted to Unicode.

Examples: 

$ pdf2txt.py -o output.html samples/naacl06-shinyama.pdf
(extract text as an HTML file whose filename is output.html)

$ pdf2txt.py -c euc-jp -D tb-rl -o output.html samples/jo.pdf
(extract a Japanese HTML file in vertical writing, CMap is required)

$ pdf2txt.py -P mypassword -o output.txt secret.pdf
(extract a text from an encrypted PDF file)

Options:

-o filename
Specifies the output file name. By default, it prints the extracted contents to stdout in text format.

-p pageno[,pageno,...]
Specifies the comma-separated list of the page numbers to be extracted. Page numbers are starting from one. By default, it extracts texts from all the pages.

-c codec
Specifies the output codec.

-t type
Specifies the output format. The following formats are currently supported.

-I image_directory
Specifies the output directory for image extraction. Currently only JPEG images are supported.

-D writing_mode
Specifies the writing mode of text outputs. Currently lr-tb, tb-rl and auto is supported.

-M char_margin
-L line_margin
-W word_margin
These are the parameters used for layout analysis. In an actual PDF file, texts might be split into several chunks in the middle of its running, depending on the authoring software. Therefore, text extraction needs to splice text chunks. In the figure below, two text chunks whose distance is closer than the char_margin (shown as M) is considered continuous and get grouped into one. Also, two lines whose distance is closer than the line_margin (L) is grouped as a text box, which is a rectangular area that contains a "cluster" of texts. Furthermore, it may be required to insert blank characters (spaces) as necessary if the distance between two words is greater than the word_margin (W), as a blank between words might not be represented as a space, but indicated by the positioning of each word.

Each value is specified not as an actual length, but as a proportion of the length to the size of each character in question. The default values are M = 1.0, L = 0.3, and W = 0.2, respectively.

→ ← M
Q u i c k b r o w n   f o x ↓
→ ← W L
j u m p s...
 ↑

-n
Suppress layout analysis.

-s scale
Specifies the output scale. Can be used in HTML format only.

-m maxpages
Specifies the maximum number of pages to extract. By default, it extracts all the pages in a document.

-P password
Provides the user password to access PDF contents.

-d
Increases the debug level.

dumppdf.py

dumppdf.py dumps the internal contents of a PDF file in pseudo-XML format. This program is primarily for debugging purposes, but it's also possible to extract some meaningful contents (such as images).

Examples: 

$ dumppdf.py -a foo.pdf
(dump all the headers and contents, except stream objects)

$ dumppdf.py -T foo.pdf
(dump the table of contents)

$ dumppdf.py -r -i6 foo.pdf > pic.jpeg
(extract a JPEG image)

Options:

-a
Instructs to dump all the objects. By default, it only prints the document trailer (like a header).

-i objno,objno, ...
Specifies PDF object IDs to display. Comma-separated IDs, or multiple -i options are accepted.

-p pageno,pageno, ...
Specifies the page number to be extracted. Comma-separated page numbers, or multiple -p options are accepted. Note that page numbers start from one, not zero.

-r (raw)
-b (binary)
-t (text)
Specifies the output format of stream contents. Because the contents of stream objects can be very large, they are omitted when none of the options above is specified.

With -r option, the "raw" stream contents are dumped without decompression. With -b option, the decompressed contents are dumped as a binary blob. With -t option, the decompressed contents are dumped in a text format, similar to repr() manner. When -r or -b option is given, no stream header is displayed for the ease of saving it to a file.

-T
Shows the table of contents.

-P password
Provides the user password to access PDF contents.

-d
Increases the debug level.

Technical Documents

TODOs

Changes

Related Projects

Terms and Conditions

(This is so-called MIT/X License)

Copyright (c) 2004-2010 Yusuke Shinyama <yusuke at cs dot nyu dot edu>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Yusuke Shinyama