2010-11-09 10:39:40 +00:00
|
|
|
"""
|
|
|
|
Miscellaneous Routines.
|
|
|
|
"""
|
2011-03-02 14:43:03 +00:00
|
|
|
import struct
|
2020-01-04 15:47:07 +00:00
|
|
|
import chardet # For str encoding detection
|
2019-10-17 10:14:02 +00:00
|
|
|
|
2019-12-29 20:20:20 +00:00
|
|
|
# from sys import maxint as INF doesn't work anymore under Python3, but PDF
|
|
|
|
# still uses 32 bits ints
|
2019-10-17 10:14:02 +00:00
|
|
|
INF = (1 << 31) - 1
|
Many changes to make pdf2txt.py work better in Py3, some in that script, others in module!
Sorry, changes should have been more atomic.
*In pdf2txt.py:*
* Re-wrote main function to use argparse instead of optparse.
* Manually tested in Py2/Py3 to get partial consistency.
* Errors abound including Tags mode, but most modes weren't working at all in Py3 anyway.
* Py2 mode *probably* unchanged, cannot find any bugs yet...
* Kept old main function for posterity, for now.
*In utils:*
* Added a few compatibility functions (some string hax required chardet, new dependency):
- make_compat_bytes(in_str)-> (py3->bytes | py2->str)
- make_compat_str(in_str)-> (str)
- compatible_encode_method(bytesorstring, encoding, erraction)-> (str)
*In pdfdevice:*
* To handle different output filetypes in Py3, injected lots of calls to new utils methods,
as well as some six.PYX checks and logic. These changes are largely responsible for
enhanced Py2/Py3 consistency.
*In converter:*
* To handle output filetypes in Py2, injected a few checks and fixes particularly around the
py2 `str.encode` method and its *assumed* usual use-analogies in Py3.
2015-05-17 20:08:57 +00:00
|
|
|
|
2019-10-17 10:14:02 +00:00
|
|
|
|
Many changes to make pdf2txt.py work better in Py3, some in that script, others in module!
Sorry, changes should have been more atomic.
*In pdf2txt.py:*
* Re-wrote main function to use argparse instead of optparse.
* Manually tested in Py2/Py3 to get partial consistency.
* Errors abound including Tags mode, but most modes weren't working at all in Py3 anyway.
* Py2 mode *probably* unchanged, cannot find any bugs yet...
* Kept old main function for posterity, for now.
*In utils:*
* Added a few compatibility functions (some string hax required chardet, new dependency):
- make_compat_bytes(in_str)-> (py3->bytes | py2->str)
- make_compat_str(in_str)-> (str)
- compatible_encode_method(bytesorstring, encoding, erraction)-> (str)
*In pdfdevice:*
* To handle different output filetypes in Py3, injected lots of calls to new utils methods,
as well as some six.PYX checks and logic. These changes are largely responsible for
enhanced Py2/Py3 consistency.
*In converter:*
* To handle output filetypes in Py2, injected a few checks and fixes particularly around the
py2 `str.encode` method and its *assumed* usual use-analogies in Py3.
2015-05-17 20:08:57 +00:00
|
|
|
def make_compat_bytes(in_str):
|
2020-01-04 15:47:07 +00:00
|
|
|
"Converts to bytes, encoding to unicode."
|
2017-05-29 07:06:09 +00:00
|
|
|
assert isinstance(in_str, str), str(type(in_str))
|
2020-01-04 15:47:07 +00:00
|
|
|
return in_str.encode()
|
Many changes to make pdf2txt.py work better in Py3, some in that script, others in module!
Sorry, changes should have been more atomic.
*In pdf2txt.py:*
* Re-wrote main function to use argparse instead of optparse.
* Manually tested in Py2/Py3 to get partial consistency.
* Errors abound including Tags mode, but most modes weren't working at all in Py3 anyway.
* Py2 mode *probably* unchanged, cannot find any bugs yet...
* Kept old main function for posterity, for now.
*In utils:*
* Added a few compatibility functions (some string hax required chardet, new dependency):
- make_compat_bytes(in_str)-> (py3->bytes | py2->str)
- make_compat_str(in_str)-> (str)
- compatible_encode_method(bytesorstring, encoding, erraction)-> (str)
*In pdfdevice:*
* To handle different output filetypes in Py3, injected lots of calls to new utils methods,
as well as some six.PYX checks and logic. These changes are largely responsible for
enhanced Py2/Py3 consistency.
*In converter:*
* To handle output filetypes in Py2, injected a few checks and fixes particularly around the
py2 `str.encode` method and its *assumed* usual use-analogies in Py3.
2015-05-17 20:08:57 +00:00
|
|
|
|
2019-10-17 10:14:02 +00:00
|
|
|
|
Many changes to make pdf2txt.py work better in Py3, some in that script, others in module!
Sorry, changes should have been more atomic.
*In pdf2txt.py:*
* Re-wrote main function to use argparse instead of optparse.
* Manually tested in Py2/Py3 to get partial consistency.
* Errors abound including Tags mode, but most modes weren't working at all in Py3 anyway.
* Py2 mode *probably* unchanged, cannot find any bugs yet...
* Kept old main function for posterity, for now.
*In utils:*
* Added a few compatibility functions (some string hax required chardet, new dependency):
- make_compat_bytes(in_str)-> (py3->bytes | py2->str)
- make_compat_str(in_str)-> (str)
- compatible_encode_method(bytesorstring, encoding, erraction)-> (str)
*In pdfdevice:*
* To handle different output filetypes in Py3, injected lots of calls to new utils methods,
as well as some six.PYX checks and logic. These changes are largely responsible for
enhanced Py2/Py3 consistency.
*In converter:*
* To handle output filetypes in Py2, injected a few checks and fixes particularly around the
py2 `str.encode` method and its *assumed* usual use-analogies in Py3.
2015-05-17 20:08:57 +00:00
|
|
|
def make_compat_str(in_str):
|
2020-01-04 15:47:07 +00:00
|
|
|
"""Converts to string, guessing encoding."""
|
|
|
|
assert isinstance(in_str, (bytes, str)), str(type(in_str))
|
|
|
|
if isinstance(in_str, bytes):
|
Many changes to make pdf2txt.py work better in Py3, some in that script, others in module!
Sorry, changes should have been more atomic.
*In pdf2txt.py:*
* Re-wrote main function to use argparse instead of optparse.
* Manually tested in Py2/Py3 to get partial consistency.
* Errors abound including Tags mode, but most modes weren't working at all in Py3 anyway.
* Py2 mode *probably* unchanged, cannot find any bugs yet...
* Kept old main function for posterity, for now.
*In utils:*
* Added a few compatibility functions (some string hax required chardet, new dependency):
- make_compat_bytes(in_str)-> (py3->bytes | py2->str)
- make_compat_str(in_str)-> (str)
- compatible_encode_method(bytesorstring, encoding, erraction)-> (str)
*In pdfdevice:*
* To handle different output filetypes in Py3, injected lots of calls to new utils methods,
as well as some six.PYX checks and logic. These changes are largely responsible for
enhanced Py2/Py3 consistency.
*In converter:*
* To handle output filetypes in Py2, injected a few checks and fixes particularly around the
py2 `str.encode` method and its *assumed* usual use-analogies in Py3.
2015-05-17 20:08:57 +00:00
|
|
|
enc = chardet.detect(in_str)
|
|
|
|
in_str = in_str.decode(enc['encoding'])
|
|
|
|
return in_str
|
|
|
|
|
2019-10-17 10:14:02 +00:00
|
|
|
|
2019-11-10 11:18:49 +00:00
|
|
|
def shorten_str(s, size):
|
|
|
|
if size < 7:
|
|
|
|
return s[:size]
|
|
|
|
if len(s) > size:
|
|
|
|
length = (size - 5) // 2
|
|
|
|
return '{} ... {}'.format(s[:length], s[-length:])
|
|
|
|
else:
|
|
|
|
return s
|
|
|
|
|
|
|
|
|
2019-12-29 20:20:20 +00:00
|
|
|
def compatible_encode_method(bytesorstring, encoding='utf-8',
|
|
|
|
erraction='ignore'):
|
|
|
|
"""When Py2 str.encode is called, it often means bytes.encode in Py3.
|
|
|
|
|
|
|
|
This does either.
|
|
|
|
"""
|
2020-01-04 15:47:07 +00:00
|
|
|
if isinstance(bytesorstring, str):
|
|
|
|
return bytesorstring
|
|
|
|
assert isinstance(bytesorstring, bytes), str(type(bytesorstring))
|
|
|
|
return bytesorstring.decode(encoding, erraction)
|
2009-01-05 04:40:50 +00:00
|
|
|
|
2019-10-17 10:14:02 +00:00
|
|
|
|
2011-06-06 15:31:03 +00:00
|
|
|
def apply_png_predictor(pred, colors, columns, bitspercomponent, data):
|
|
|
|
if bitspercomponent != 8:
|
|
|
|
# unsupported
|
2016-09-23 12:31:31 +00:00
|
|
|
raise ValueError("Unsupported `bitspercomponent': %d" %
|
|
|
|
bitspercomponent)
|
|
|
|
nbytes = colors * columns * bitspercomponent // 8
|
2014-06-30 10:05:56 +00:00
|
|
|
buf = b''
|
|
|
|
line0 = b'\x00' * columns
|
2019-10-17 10:14:02 +00:00
|
|
|
for i in range(0, len(data), nbytes + 1):
|
2014-03-24 10:57:05 +00:00
|
|
|
ft = data[i]
|
2011-06-06 15:31:03 +00:00
|
|
|
i += 1
|
2019-10-17 10:14:02 +00:00
|
|
|
line1 = data[i:i + nbytes]
|
2014-06-30 10:05:56 +00:00
|
|
|
line2 = b''
|
2014-09-16 20:59:29 +00:00
|
|
|
if ft == 0:
|
2011-06-06 15:31:03 +00:00
|
|
|
# PNG none
|
2014-03-24 10:57:05 +00:00
|
|
|
line2 += line1
|
2014-09-16 20:59:29 +00:00
|
|
|
elif ft == 1:
|
2011-06-06 15:46:33 +00:00
|
|
|
# PNG sub (UNTESTED)
|
|
|
|
c = 0
|
|
|
|
for b in line1:
|
2019-10-17 10:14:02 +00:00
|
|
|
c = (c + b) & 255
|
2020-01-04 15:47:07 +00:00
|
|
|
line2 += bytes((c,))
|
2014-09-16 20:59:29 +00:00
|
|
|
elif ft == 2:
|
2011-06-06 15:31:03 +00:00
|
|
|
# PNG up
|
2013-11-07 08:35:04 +00:00
|
|
|
for (a, b) in zip(line0, line1):
|
2019-10-17 10:14:02 +00:00
|
|
|
c = (a + b) & 255
|
2020-01-04 15:47:07 +00:00
|
|
|
line2 += bytes((c,))
|
2014-09-16 20:59:29 +00:00
|
|
|
elif ft == 3:
|
2011-06-06 15:46:33 +00:00
|
|
|
# PNG average (UNTESTED)
|
|
|
|
c = 0
|
2013-11-07 08:35:04 +00:00
|
|
|
for (a, b) in zip(line0, line1):
|
2019-10-17 10:14:02 +00:00
|
|
|
c = ((c + a + b) // 2) & 255
|
2020-01-04 15:47:07 +00:00
|
|
|
line2 += bytes((c,))
|
2011-06-06 15:31:03 +00:00
|
|
|
else:
|
|
|
|
# unsupported
|
2016-09-23 12:31:31 +00:00
|
|
|
raise ValueError("Unsupported predictor value: %d" % ft)
|
2014-03-24 10:57:05 +00:00
|
|
|
buf += line2
|
|
|
|
line0 = line2
|
2011-06-06 15:31:03 +00:00
|
|
|
return buf
|
|
|
|
|
|
|
|
|
2019-10-17 10:14:02 +00:00
|
|
|
# Matrix operations
|
2009-01-10 09:14:46 +00:00
|
|
|
MATRIX_IDENTITY = (1, 0, 0, 1, 0, 0)
|
|
|
|
|
2013-11-07 08:35:04 +00:00
|
|
|
|
2014-06-15 03:12:13 +00:00
|
|
|
def mult_matrix(m1, m0):
|
|
|
|
(a1, b1, c1, d1, e1, f1) = m1
|
|
|
|
(a0, b0, c0, d0, e0, f0) = m0
|
2011-02-27 04:09:12 +00:00
|
|
|
"""Returns the multiplication of two matrices."""
|
2019-10-17 10:14:02 +00:00
|
|
|
return (a0 * a1 + c0 * b1, b0 * a1 + d0 * b1,
|
|
|
|
a0 * c1 + c0 * d1, b0 * c1 + d0 * d1,
|
|
|
|
a0 * e1 + c0 * f1 + e0, b0 * e1 + d0 * f1 + f0)
|
2009-01-05 04:40:50 +00:00
|
|
|
|
2009-10-24 04:41:59 +00:00
|
|
|
|
2014-06-15 03:12:13 +00:00
|
|
|
def translate_matrix(m, v):
|
2013-11-07 08:35:04 +00:00
|
|
|
"""Translates a matrix by (x, y)."""
|
2014-06-15 03:12:13 +00:00
|
|
|
(a, b, c, d, e, f) = m
|
|
|
|
(x, y) = v
|
2019-10-17 10:14:02 +00:00
|
|
|
return a, b, c, d, x * a + y * c + e, x * b + y * d + f
|
2013-11-07 08:35:04 +00:00
|
|
|
|
|
|
|
|
2014-06-15 03:12:13 +00:00
|
|
|
def apply_matrix_pt(m, v):
|
|
|
|
(a, b, c, d, e, f) = m
|
|
|
|
(x, y) = v
|
2011-02-27 04:09:12 +00:00
|
|
|
"""Applies a matrix to a point."""
|
2019-10-17 10:14:02 +00:00
|
|
|
return a * x + c * y + e, b * x + d * y + f
|
2009-01-05 04:40:50 +00:00
|
|
|
|
2013-11-07 08:35:04 +00:00
|
|
|
|
2014-06-15 03:12:13 +00:00
|
|
|
def apply_matrix_norm(m, v):
|
2011-02-27 04:09:12 +00:00
|
|
|
"""Equivalent to apply_matrix_pt(M, (p,q)) - apply_matrix_pt(M, (0,0))"""
|
2014-06-15 03:12:13 +00:00
|
|
|
(a, b, c, d, e, f) = m
|
|
|
|
(p, q) = v
|
2019-10-17 10:14:02 +00:00
|
|
|
return a * p + c * q, b * p + d * q
|
2009-01-05 04:40:50 +00:00
|
|
|
|
2007-12-31 03:41:45 +00:00
|
|
|
|
2019-10-17 10:14:02 +00:00
|
|
|
# Utility functions
|
2009-07-11 15:28:12 +00:00
|
|
|
|
2014-03-28 08:50:59 +00:00
|
|
|
def isnumber(x):
|
2020-01-07 20:59:13 +00:00
|
|
|
return isinstance(x, (int, float))
|
2014-03-28 08:50:59 +00:00
|
|
|
|
2019-10-17 10:14:02 +00:00
|
|
|
|
2011-02-27 03:56:28 +00:00
|
|
|
def uniq(objs):
|
2011-02-27 04:09:12 +00:00
|
|
|
"""Eliminates duplicated elements."""
|
2011-02-27 03:56:28 +00:00
|
|
|
done = set()
|
|
|
|
for obj in objs:
|
2013-11-07 08:35:04 +00:00
|
|
|
if obj in done:
|
|
|
|
continue
|
2011-02-27 03:56:28 +00:00
|
|
|
done.add(obj)
|
|
|
|
yield obj
|
|
|
|
return
|
|
|
|
|
2013-11-07 08:35:04 +00:00
|
|
|
|
2011-03-01 11:47:20 +00:00
|
|
|
def fsplit(pred, objs):
|
|
|
|
"""Split a list into two classes according to the predicate."""
|
|
|
|
t = []
|
|
|
|
f = []
|
|
|
|
for obj in objs:
|
|
|
|
if pred(obj):
|
|
|
|
t.append(obj)
|
|
|
|
else:
|
|
|
|
f.append(obj)
|
2019-10-17 10:14:02 +00:00
|
|
|
return t, f
|
2013-11-07 08:35:04 +00:00
|
|
|
|
2011-03-01 11:47:20 +00:00
|
|
|
|
2011-02-27 03:56:28 +00:00
|
|
|
def drange(v0, v1, d):
|
2011-02-27 04:09:12 +00:00
|
|
|
"""Returns a discrete range."""
|
2019-10-17 10:14:02 +00:00
|
|
|
return range(int(v0) // d, int(v1 + d) // d)
|
2011-02-27 03:56:28 +00:00
|
|
|
|
2013-11-07 08:35:04 +00:00
|
|
|
|
2010-10-17 05:13:39 +00:00
|
|
|
def get_bound(pts):
|
2011-02-27 04:09:12 +00:00
|
|
|
"""Compute a minimal rectangle that covers all the points."""
|
2010-10-17 05:13:39 +00:00
|
|
|
(x0, y0, x1, y1) = (INF, INF, -INF, -INF)
|
2013-11-07 08:35:04 +00:00
|
|
|
for (x, y) in pts:
|
2010-10-17 05:13:39 +00:00
|
|
|
x0 = min(x0, x)
|
|
|
|
y0 = min(y0, y)
|
|
|
|
x1 = max(x1, x)
|
|
|
|
y1 = max(y1, y)
|
2019-10-17 10:14:02 +00:00
|
|
|
return x0, y0, x1, y1
|
2013-11-07 08:35:04 +00:00
|
|
|
|
2010-10-17 05:13:39 +00:00
|
|
|
|
2009-07-11 15:28:12 +00:00
|
|
|
def pick(seq, func, maxobj=None):
|
2011-02-27 04:09:12 +00:00
|
|
|
"""Picks the object obj where func(obj) has the highest value."""
|
2009-10-24 04:41:59 +00:00
|
|
|
maxscore = None
|
|
|
|
for obj in seq:
|
|
|
|
score = func(obj)
|
2009-11-06 15:10:29 +00:00
|
|
|
if maxscore is None or maxscore < score:
|
2013-11-07 08:35:04 +00:00
|
|
|
(maxscore, maxobj) = (score, obj)
|
2009-10-24 04:41:59 +00:00
|
|
|
return maxobj
|
2009-07-11 15:28:12 +00:00
|
|
|
|
2013-11-07 08:35:04 +00:00
|
|
|
|
2007-12-31 03:41:45 +00:00
|
|
|
def choplist(n, seq):
|
2011-02-27 04:09:12 +00:00
|
|
|
"""Groups every n elements of the list."""
|
2009-10-24 04:41:59 +00:00
|
|
|
r = []
|
|
|
|
for x in seq:
|
|
|
|
r.append(x)
|
|
|
|
if len(r) == n:
|
|
|
|
yield tuple(r)
|
|
|
|
r = []
|
|
|
|
return
|
2007-12-31 03:41:45 +00:00
|
|
|
|
2013-11-07 08:35:04 +00:00
|
|
|
|
2007-12-31 03:41:45 +00:00
|
|
|
def nunpack(s, default=0):
|
2017-02-06 13:57:01 +00:00
|
|
|
"""Unpacks 1 to 4 or 8 byte integers (big endian)."""
|
2019-12-29 20:20:20 +00:00
|
|
|
length = len(s)
|
|
|
|
if not length:
|
2009-10-24 04:41:59 +00:00
|
|
|
return default
|
2019-12-29 20:20:20 +00:00
|
|
|
elif length == 1:
|
2009-10-24 04:41:59 +00:00
|
|
|
return ord(s)
|
2019-12-29 20:20:20 +00:00
|
|
|
elif length == 2:
|
2011-03-02 14:43:03 +00:00
|
|
|
return struct.unpack('>H', s)[0]
|
2019-12-29 20:20:20 +00:00
|
|
|
elif length == 3:
|
2019-10-17 10:14:02 +00:00
|
|
|
return struct.unpack('>L', b'\x00' + s)[0]
|
2019-12-29 20:20:20 +00:00
|
|
|
elif length == 4:
|
2011-03-02 14:43:03 +00:00
|
|
|
return struct.unpack('>L', s)[0]
|
2019-12-29 20:20:20 +00:00
|
|
|
elif length == 8:
|
2017-02-06 13:57:01 +00:00
|
|
|
return struct.unpack('>Q', s)[0]
|
2009-10-24 04:41:59 +00:00
|
|
|
else:
|
2019-12-29 20:20:20 +00:00
|
|
|
raise TypeError('invalid length: %d' % length)
|
2009-02-01 15:01:32 +00:00
|
|
|
|
2013-11-07 08:35:04 +00:00
|
|
|
|
2020-01-04 15:47:07 +00:00
|
|
|
PDFDocEncoding = ''.join(chr(x) for x in (
|
2013-11-07 08:35:04 +00:00
|
|
|
0x0000, 0x0001, 0x0002, 0x0003, 0x0004, 0x0005, 0x0006, 0x0007,
|
|
|
|
0x0008, 0x0009, 0x000a, 0x000b, 0x000c, 0x000d, 0x000e, 0x000f,
|
|
|
|
0x0010, 0x0011, 0x0012, 0x0013, 0x0014, 0x0015, 0x0017, 0x0017,
|
|
|
|
0x02d8, 0x02c7, 0x02c6, 0x02d9, 0x02dd, 0x02db, 0x02da, 0x02dc,
|
|
|
|
0x0020, 0x0021, 0x0022, 0x0023, 0x0024, 0x0025, 0x0026, 0x0027,
|
|
|
|
0x0028, 0x0029, 0x002a, 0x002b, 0x002c, 0x002d, 0x002e, 0x002f,
|
|
|
|
0x0030, 0x0031, 0x0032, 0x0033, 0x0034, 0x0035, 0x0036, 0x0037,
|
|
|
|
0x0038, 0x0039, 0x003a, 0x003b, 0x003c, 0x003d, 0x003e, 0x003f,
|
|
|
|
0x0040, 0x0041, 0x0042, 0x0043, 0x0044, 0x0045, 0x0046, 0x0047,
|
|
|
|
0x0048, 0x0049, 0x004a, 0x004b, 0x004c, 0x004d, 0x004e, 0x004f,
|
|
|
|
0x0050, 0x0051, 0x0052, 0x0053, 0x0054, 0x0055, 0x0056, 0x0057,
|
|
|
|
0x0058, 0x0059, 0x005a, 0x005b, 0x005c, 0x005d, 0x005e, 0x005f,
|
|
|
|
0x0060, 0x0061, 0x0062, 0x0063, 0x0064, 0x0065, 0x0066, 0x0067,
|
|
|
|
0x0068, 0x0069, 0x006a, 0x006b, 0x006c, 0x006d, 0x006e, 0x006f,
|
|
|
|
0x0070, 0x0071, 0x0072, 0x0073, 0x0074, 0x0075, 0x0076, 0x0077,
|
|
|
|
0x0078, 0x0079, 0x007a, 0x007b, 0x007c, 0x007d, 0x007e, 0x0000,
|
|
|
|
0x2022, 0x2020, 0x2021, 0x2026, 0x2014, 0x2013, 0x0192, 0x2044,
|
|
|
|
0x2039, 0x203a, 0x2212, 0x2030, 0x201e, 0x201c, 0x201d, 0x2018,
|
|
|
|
0x2019, 0x201a, 0x2122, 0xfb01, 0xfb02, 0x0141, 0x0152, 0x0160,
|
|
|
|
0x0178, 0x017d, 0x0131, 0x0142, 0x0153, 0x0161, 0x017e, 0x0000,
|
|
|
|
0x20ac, 0x00a1, 0x00a2, 0x00a3, 0x00a4, 0x00a5, 0x00a6, 0x00a7,
|
|
|
|
0x00a8, 0x00a9, 0x00aa, 0x00ab, 0x00ac, 0x0000, 0x00ae, 0x00af,
|
|
|
|
0x00b0, 0x00b1, 0x00b2, 0x00b3, 0x00b4, 0x00b5, 0x00b6, 0x00b7,
|
|
|
|
0x00b8, 0x00b9, 0x00ba, 0x00bb, 0x00bc, 0x00bd, 0x00be, 0x00bf,
|
|
|
|
0x00c0, 0x00c1, 0x00c2, 0x00c3, 0x00c4, 0x00c5, 0x00c6, 0x00c7,
|
|
|
|
0x00c8, 0x00c9, 0x00ca, 0x00cb, 0x00cc, 0x00cd, 0x00ce, 0x00cf,
|
|
|
|
0x00d0, 0x00d1, 0x00d2, 0x00d3, 0x00d4, 0x00d5, 0x00d6, 0x00d7,
|
|
|
|
0x00d8, 0x00d9, 0x00da, 0x00db, 0x00dc, 0x00dd, 0x00de, 0x00df,
|
|
|
|
0x00e0, 0x00e1, 0x00e2, 0x00e3, 0x00e4, 0x00e5, 0x00e6, 0x00e7,
|
|
|
|
0x00e8, 0x00e9, 0x00ea, 0x00eb, 0x00ec, 0x00ed, 0x00ee, 0x00ef,
|
|
|
|
0x00f0, 0x00f1, 0x00f2, 0x00f3, 0x00f4, 0x00f5, 0x00f6, 0x00f7,
|
|
|
|
0x00f8, 0x00f9, 0x00fa, 0x00fb, 0x00fc, 0x00fd, 0x00fe, 0x00ff,
|
2009-02-01 15:01:32 +00:00
|
|
|
))
|
2013-11-07 08:35:04 +00:00
|
|
|
|
|
|
|
|
2009-02-01 15:01:32 +00:00
|
|
|
def decode_text(s):
|
2011-02-27 04:09:12 +00:00
|
|
|
"""Decodes a PDFDocEncoding string to Unicode."""
|
2014-06-30 10:05:56 +00:00
|
|
|
if s.startswith(b'\xfe\xff'):
|
2020-01-04 15:47:07 +00:00
|
|
|
return str(s[2:], 'utf-16be', 'ignore')
|
2009-10-24 04:41:59 +00:00
|
|
|
else:
|
2017-02-06 13:57:01 +00:00
|
|
|
return ''.join(PDFDocEncoding[c] for c in s)
|
2013-11-07 08:35:04 +00:00
|
|
|
|
2009-05-15 14:25:32 +00:00
|
|
|
|
|
|
|
def enc(x, codec='ascii'):
|
2011-02-27 04:09:12 +00:00
|
|
|
"""Encodes a string for SGML/XML/HTML"""
|
2020-01-04 15:47:07 +00:00
|
|
|
if isinstance(x, bytes):
|
2016-12-14 16:34:16 +00:00
|
|
|
return ''
|
2019-12-29 20:20:20 +00:00
|
|
|
x = x.replace('&', '&').replace('>', '>').replace('<', '<') \
|
|
|
|
.replace('"', '"')
|
2014-09-16 20:57:00 +00:00
|
|
|
if codec:
|
|
|
|
x = x.encode(codec, 'xmlcharrefreplace')
|
|
|
|
return x
|
2010-01-01 03:09:26 +00:00
|
|
|
|
2013-11-07 08:35:04 +00:00
|
|
|
|
2014-06-15 03:12:13 +00:00
|
|
|
def bbox2str(bbox):
|
|
|
|
(x0, y0, x1, y1) = bbox
|
2020-01-04 15:47:07 +00:00
|
|
|
return '{:.3f},{:.3f},{:.3f},{:.3f}'.format(x0, y0, x1, y1)
|
2010-01-31 02:09:28 +00:00
|
|
|
|
2013-11-07 08:35:04 +00:00
|
|
|
|
2014-06-15 03:12:13 +00:00
|
|
|
def matrix2str(m):
|
|
|
|
(a, b, c, d, e, f) = m
|
2020-01-04 15:47:07 +00:00
|
|
|
return '[{:.2f},{:.2f},{:.2f},{:.2f}, ({:.2f},{:.2f})]'\
|
|
|
|
.format(a, b, c, d, e, f)
|
2010-02-27 03:59:25 +00:00
|
|
|
|
2019-10-17 10:14:02 +00:00
|
|
|
|
2018-02-16 14:49:12 +00:00
|
|
|
def vecBetweenBoxes(obj1, obj2):
|
|
|
|
"""A distance function between two TextBoxes.
|
|
|
|
|
|
|
|
Consider the bounding rectangle for obj1 and obj2.
|
2019-12-29 20:20:20 +00:00
|
|
|
Return vector between 2 boxes boundaries if they don't overlap, otherwise
|
|
|
|
returns vector betweeen boxes centers
|
|
|
|
|
2018-02-16 14:49:12 +00:00
|
|
|
+------+..........+ (x1, y1)
|
|
|
|
| obj1 | :
|
|
|
|
+------+www+------+
|
|
|
|
: | obj2 |
|
|
|
|
(x0, y0) +..........+------+
|
|
|
|
"""
|
|
|
|
(x0, y0) = (min(obj1.x0, obj2.x0), min(obj1.y0, obj2.y0))
|
|
|
|
(x1, y1) = (max(obj1.x1, obj2.x1), max(obj1.y1, obj2.y1))
|
2019-10-17 10:14:02 +00:00
|
|
|
(ow, oh) = (x1 - x0, y1 - y0)
|
|
|
|
(iw, ih) = (ow - obj1.width - obj2.width, oh - obj1.height - obj2.height)
|
|
|
|
if iw < 0 and ih < 0:
|
2018-02-16 14:49:12 +00:00
|
|
|
# if one is inside another we compute euclidean distance
|
2019-10-17 10:14:02 +00:00
|
|
|
(xc1, yc1) = ((obj1.x0 + obj1.x1) / 2, (obj1.y0 + obj1.y1) / 2)
|
|
|
|
(xc2, yc2) = ((obj2.x0 + obj2.x1) / 2, (obj2.y0 + obj2.y1) / 2)
|
|
|
|
return xc1 - xc2, yc1 - yc2
|
2018-02-16 14:49:12 +00:00
|
|
|
else:
|
2019-10-17 10:14:02 +00:00
|
|
|
return max(0, iw), max(0, ih)
|
|
|
|
|
|
|
|
|
2020-01-04 15:47:07 +00:00
|
|
|
class Plane:
|
2019-10-17 10:14:02 +00:00
|
|
|
"""A set-like data structure for objects placed on a plane.
|
|
|
|
|
|
|
|
Can efficiently find objects in a certain rectangular area.
|
|
|
|
It maintains two parallel lists of objects, each of
|
|
|
|
which is sorted by its x or y coordinate.
|
|
|
|
"""
|
2010-11-09 10:40:05 +00:00
|
|
|
|
2013-10-22 09:24:39 +00:00
|
|
|
def __init__(self, bbox, gridsize=50):
|
2019-10-17 10:14:02 +00:00
|
|
|
self._seq = [] # preserve the object order.
|
2012-06-22 19:36:33 +00:00
|
|
|
self._objs = set()
|
2011-05-14 05:16:40 +00:00
|
|
|
self._grid = {}
|
2011-02-27 03:56:28 +00:00
|
|
|
self.gridsize = gridsize
|
2013-10-22 09:24:39 +00:00
|
|
|
(self.x0, self.y0, self.x1, self.y1) = bbox
|
2010-11-09 10:40:05 +00:00
|
|
|
|
|
|
|
def __repr__(self):
|
2019-10-17 10:14:02 +00:00
|
|
|
return '<Plane objs=%r>' % list(self)
|
2010-11-09 10:40:05 +00:00
|
|
|
|
2011-05-14 05:16:40 +00:00
|
|
|
def __iter__(self):
|
2019-10-17 10:14:02 +00:00
|
|
|
return (obj for obj in self._seq if obj in self._objs)
|
2011-05-14 05:16:40 +00:00
|
|
|
|
|
|
|
def __len__(self):
|
|
|
|
return len(self._objs)
|
|
|
|
|
|
|
|
def __contains__(self, obj):
|
|
|
|
return obj in self._objs
|
|
|
|
|
2014-06-15 03:12:13 +00:00
|
|
|
def _getrange(self, bbox):
|
|
|
|
(x0, y0, x1, y1) = bbox
|
2019-10-17 10:14:02 +00:00
|
|
|
if x1 <= self.x0 or self.x1 <= x0 or y1 <= self.y0 or self.y1 <= y0:
|
|
|
|
return
|
2013-10-22 09:24:39 +00:00
|
|
|
x0 = max(self.x0, x0)
|
|
|
|
y0 = max(self.y0, y0)
|
|
|
|
x1 = min(self.x1, x1)
|
|
|
|
y1 = min(self.y1, y1)
|
2019-10-17 10:14:02 +00:00
|
|
|
for grid_y in drange(y0, y1, self.gridsize):
|
|
|
|
for grid_x in drange(x0, x1, self.gridsize):
|
|
|
|
yield (grid_x, grid_y)
|
2013-10-22 09:24:39 +00:00
|
|
|
|
|
|
|
def extend(self, objs):
|
|
|
|
for obj in objs:
|
|
|
|
self.add(obj)
|
2013-11-07 07:14:53 +00:00
|
|
|
|
2010-11-09 10:40:05 +00:00
|
|
|
def add(self, obj):
|
2019-10-17 10:14:02 +00:00
|
|
|
"""place an object."""
|
2011-02-27 03:56:28 +00:00
|
|
|
for k in self._getrange((obj.x0, obj.y0, obj.x1, obj.y1)):
|
2011-05-14 05:16:40 +00:00
|
|
|
if k not in self._grid:
|
2011-02-27 03:56:28 +00:00
|
|
|
r = []
|
2011-05-14 05:16:40 +00:00
|
|
|
self._grid[k] = r
|
2011-02-27 03:56:28 +00:00
|
|
|
else:
|
2011-05-14 05:16:40 +00:00
|
|
|
r = self._grid[k]
|
2011-02-27 03:56:28 +00:00
|
|
|
r.append(obj)
|
2014-06-14 05:44:53 +00:00
|
|
|
self._seq.append(obj)
|
2012-06-22 19:36:33 +00:00
|
|
|
self._objs.add(obj)
|
2010-11-09 10:40:05 +00:00
|
|
|
|
2011-04-21 13:07:04 +00:00
|
|
|
def remove(self, obj):
|
2019-10-17 10:14:02 +00:00
|
|
|
"""displace an object."""
|
2011-04-21 13:07:04 +00:00
|
|
|
for k in self._getrange((obj.x0, obj.y0, obj.x1, obj.y1)):
|
|
|
|
try:
|
2011-05-14 05:16:40 +00:00
|
|
|
self._grid[k].remove(obj)
|
2011-04-21 13:07:04 +00:00
|
|
|
except (KeyError, ValueError):
|
|
|
|
pass
|
2011-05-14 05:16:40 +00:00
|
|
|
self._objs.remove(obj)
|
2011-04-21 13:07:04 +00:00
|
|
|
|
2014-06-15 03:12:13 +00:00
|
|
|
def find(self, bbox):
|
2019-10-17 10:14:02 +00:00
|
|
|
"""finds objects that are in a certain area."""
|
2014-06-15 03:12:13 +00:00
|
|
|
(x0, y0, x1, y1) = bbox
|
2011-05-14 05:16:40 +00:00
|
|
|
done = set()
|
2014-06-15 03:12:13 +00:00
|
|
|
for k in self._getrange(bbox):
|
2013-11-07 08:35:04 +00:00
|
|
|
if k not in self._grid:
|
|
|
|
continue
|
2011-05-14 05:16:40 +00:00
|
|
|
for obj in self._grid[k]:
|
2013-11-07 08:35:04 +00:00
|
|
|
if obj in done:
|
|
|
|
continue
|
2011-05-14 05:16:40 +00:00
|
|
|
done.add(obj)
|
2019-12-29 20:20:20 +00:00
|
|
|
if obj.x1 <= x0 or x1 <= obj.x0 or obj.y1 <= y0 \
|
|
|
|
or y1 <= obj.y0:
|
2013-11-07 08:35:04 +00:00
|
|
|
continue
|
2011-05-14 16:22:19 +00:00
|
|
|
yield obj
|