[Here's an on-charter post for a change...]
I was given some scans of sheet music from an iPad vict^H^H^H^Huser who couldn't find the settings (if it has any) for its scanning function.
As a result, she wound up with a 9-page PDF file that takes up over
30 megabytes. (When I scan a similar document on my machine the file
is about one megabyte.) A bit of digging came up with utilities that
tell me that her scan is in 24-bit colour at some ungodly resolution
(using JPEG to keep it from being even bigger).
I've been looking at utilities like Ghostscript, but the documentation
is making my eyes glaze over. Is there a guide that can help me
re-sample the file, say to 300 dpi in monochrome?
[Here's an on-charter post for a change...]
I was given some scans of sheet music from an iPad vict^H^H^H^Huser who couldn't find the settings (if it has any) for its scanning function.
As a result, she wound up with a 9-page PDF file that takes up over
30 megabytes. (When I scan a similar document on my machine the file
is about one megabyte.) A bit of digging came up with utilities that
tell me that her scan is in 24-bit colour at some ungodly resolution
(using JPEG to keep it from being even bigger).
I've been looking at utilities like Ghostscript, but the documentation
is making my eyes glaze over. Is there a guide that can help me
re-sample the file, say to 300 dpi in monochrome?
I suppose the alternative would be to have the user find someone
who could help her beat macOS into submission, but that might
wind up more like that episode of "The Big Bang Theory":
Leonard: Oh, screw the roommate agreement.
Sheldon: No! You don't screw the roommate agreement.
The roommate agreement screws _you_!
aTdHvAaNnKcSe...
I've been looking at utilities like Ghostscript, but the
documentation is making my eyes glaze over.
## 4.1 Creating Black-and-White PDF Documents
Official documents, many university lecture notes, and numerous books are published in black and white. This makes it possible to significantly reduce the size of the resulting PDF document.
The process of creating a PDF document consists of several steps:
1. Converting PNG files to PBM;
2. Converting PBM files to DJVU;
3. Combining all DJVU files into a single DJVU file;
4. Converting the resulting DJVU file into the final PDF file.
Conversion to DJVU is necessary because this format currently provides the best compression for scanned images containing text. Conversion to PBM is necessary because this is the input format accepted by the `cjb2` program, which generates DJVU files.
```bash
# To convert PNG files to PBM, call the following
# command in a loop:
convert "$lPNG" "$lPBM"
```
Of course, you must assign the names of the files being converted to the `lPNG` and `lPBM` variables.
```bash
# To convert PBM files to DJVU, call cjb2
# (Simple DjVuBitonal encoder) in a loop:
cjb2 -clean "$lPBM" "$lDJVU"
```
Of course, you must assign the names of the files being converted to the `lPBM` and `lDJVU` variables.
```bash
# To merge individual DJVU files into a single
# temporary file named tmp.djvu:
djvm -c "./tmp.djvu" $(find . -maxdepth
1 -type f -iname '*.djvu' | nsort)
```
`nsort` is my own utility, which sorts filenames by their numeric values rather than by their ASCII codes.
As an alternative, you can use the standard system `sort` command, provided that you use filenames with leading zeros, e.g. `00001.png`, `00002.png`, and so on.
```bash
# To convert the merged DJVU file to PDF:
ddjvu -format=pdf -quality=150
"./tmp.djvu" "$gNazwaPDF"
```
`-quality=150` provides the best quality.
For testing purposes, I scanned the first four pages of an article entitled "The Return of Kakuro", from issue 77 (1/2025) of the two-colour magazine *?wiat Matematyki*.
To demonstrate the effectiveness of the compression, I also combined the PNG files into a PDF using the following command:
```bash
convert $(find . -maxdepth 1 -type f -
iname '*.png' | nsort) 'Powrąt Kakuro.v2.pdf'
```
The resulting files had the following sizes:
```text
256975 'Powrąt Kakuro.pdf'
637496 'Powrąt Kakuro.v2.pdf'
154399 'Powrąt Kakuro.djvu'
```
This means that the resulting `Powrąt Kakuro.pdf` file was 166% larger than the DJVU version, while the lossless version was 248% larger.
[Here's an on-charter post for a change...]
I was given some scans of sheet music from an iPad vict^H^H^H^Huser who couldn't find the settings (if it has any) for its scanning function.
As a result, she wound up with a 9-page PDF file that takes up over
30 megabytes. (When I scan a similar document on my machine the file
is about one megabyte.) A bit of digging came up with utilities that
tell me that her scan is in 24-bit colour at some ungodly resolution
(using JPEG to keep it from being even bigger).
I've been looking at utilities like Ghostscript, but the documentation
is making my eyes glaze over. Is there a guide that can help me
re-sample the file, say to 300 dpi in monochrome?
[Here's an on-charter post for a change...]
I was given some scans of sheet music from an iPad vict^H^H^H^Huser who couldn't find the settings (if it has any) for its scanning function.
As a result, she wound up with a 9-page PDF file that takes up over
30 megabytes. (When I scan a similar document on my machine the file
is about one megabyte.) A bit of digging came up with utilities that
tell me that her scan is in 24-bit colour at some ungodly resolution
(using JPEG to keep it from being even bigger).
I've been looking at utilities like Ghostscript, but the documentation
is making my eyes glaze over. Is there a guide that can help me
re-sample the file, say to 300 dpi in monochrome?
I was given some scans of sheet music from an iPad vict^H^H^H^Huser who couldn't find the settings (if it has any) for its scanning function.
As a result, she wound up with a 9-page PDF file that takes up over
30 megabytes. (When I scan a similar document on my machine the file
is about one megabyte.) A bit of digging came up with utilities that
tell me that her scan is in 24-bit colour at some ungodly resolution
(using JPEG to keep it from being even bigger).
I've been looking at utilities like Ghostscript, but the documentation
is making my eyes glaze over. Is there a guide that can help me
re-sample the file, say to 300 dpi in monochrome?
On 2026-11-15 01:56, Charlie Gibbs wrote:
[Here's an on-charter post for a change...]
I was given some scans of sheet music from an iPad vict^H^H^H^Huser who
couldn't find the settings (if it has any) for its scanning function.
As a result, she wound up with a 9-page PDF file that takes up over
30 megabytes. (When I scan a similar document on my machine the file
is about one megabyte.) A bit of digging came up with utilities that
tell me that her scan is in 24-bit colour at some ungodly resolution
(using JPEG to keep it from being even bigger).
I've been looking at utilities like Ghostscript, but the documentation
is making my eyes glaze over. Is there a guide that can help me
re-sample the file, say to 300 dpi in monochrome?
Convert to some bitmap image format, file per page (png, for instance) resample using gimp, or magick
convert back to PDF, if that is the goal.
Hint: don't fight documentation. Ask an AI to guide you on the use of ghostscript (if that is the tool you want to use). AIs are good at
this.
(no, I did not ask an AI for this post)
On 2026-11-15, Carlos E.R. wrote:
On 2026-11-15 01:56, Charlie Gibbs wrote:
[Here's an on-charter post for a change...]
I was given some scans of sheet music from an iPad vict^H^H^H^Huser who
couldn't find the settings (if it has any) for its scanning function.
As a result, she wound up with a 9-page PDF file that takes up over
30 megabytes. (When I scan a similar document on my machine the file
is about one megabyte.) A bit of digging came up with utilities that
tell me that her scan is in 24-bit colour at some ungodly resolution
(using JPEG to keep it from being even bigger).
I've been looking at utilities like Ghostscript, but the documentation
is making my eyes glaze over. Is there a guide that can help me
re-sample the file, say to 300 dpi in monochrome?
Convert to some bitmap image format, file per page (png, for instance)
resample using gimp, or magick
For this I stress my recommendation of poppler's pdfimages. If I'm not mistaken, it will extract the image streams, not convert them. If all it
has is one image per page, that's probably the best to avoid loss at
this stage.
Thanks, everyone, for your suggestions. Here's a history of my
various attempts. To keep things simple, I extracted the first
page of the 11-page original file (32,468,435 bytes) as follows:
$ pdftk hugefile.pdf cat 1 output testfile.pdf
This resulted in a file of 1,520,696 bytes; the reduction in size is
a lot more than 11 to 1, so pdftk must be doing some magic of its own.
$ pdfimages -list testfile.pdf
page num type width height color comp bpc enc interp object ID x-ppi y-ppi size ratio
--------------------------------------------------------------------------------------------
1 0 image 1928 2754 icc 3 8 jpeg yes 6 0 72 72 1481K 9.5%
This indicates colour (icc) with three 8-bit components per pixel.
$ identify testfile.pdf
testfile.pdf PDF 1944x2772 1944x2772+0+0 16-bit sRGB 9615B 0.010u 0:00.000
Now let's have GhostScript convert the file to grayscale,
using a suggestion from Nuno Silva:
$ gs -sDEVICE=pdfwrite -sColorConversionStrategy=Gray \
-sProcessColorModel=DeviceGray -sOutputFile=testfile2.pdf testfile.pdf
This converted the file to grayscale (8 bits per pixel, still with
JPEG compression; its size is now 359,415 bytes.
Following the URL given by Lawrence D'Oliveiro <https://ghostscript.readthedocs.io/gs10.08.0/Devices.html>
gave a bewildering array of possibilities (the GhostScript
command "devicenames ==" is even more intimidating).
Changing the device from pdfwrite to pngmono got the size down
to 112,386 bytes. The ImageMagick utility "identify" says it's
8-bit grayscale.
We're getting the file size down to something more reasonable.
However, contrast isn't all that good. Let's see whether we
can go for monochrome.
Working with Rich's suggestions, I ran the following:
$ gs -sDEVICE=pgmraw -sColorConversionStrategy=Gray \
-sProcessColorModel=DeviceGray -sOutputFile=testfile3.pgm testfile.pdf
$ pgmtopbm -threshold -value 0.82 testfile3.pgm | pnmtotiff -g4 >testfile3.tif
$ tiff2pdf -o testfile3.pdf testfile3.tif
Bingo! I wound up with a crisp PDF file whose size is only 48,071
bytes: a 30-to-1 reduction. This size is typical of what I get
when I do my own scanning. And by tweaking the threshold value
from 0.82 to 0.9 I was able to bring out penciled notes without
getting black blotches everywhere.
Adding the capability to handle multi-page documents uncovered a
few problems. The sizes of the images seem to vary from page to
page, and some some pages have shadows that make it difficult to
make pencil marks visible without getting those big black blotches
I mentioned earlier. If I could persuade the person doing the
scanning to be careful to avoid shadows, my final script should
be able to do a wonderful job of cleaning up such files. This
might not be an option, though - most people seem unable or
unwilling to take the necessary care. At least the images
aren't keystoned...
Thanks, everyone, for your help!
On 11/15/26 18:31, Charlie Gibbs wrote:
Thanks, everyone, for your suggestions. Here's a history of my
various attempts.
Umm ... a two/three/four-stage CL 'solution' to
what SHOULD be easy ?
PDFs have a very useful place, but for SOME
purposes they're an over-complex disaster.
On 2026-11-16, c186282 <c186282@nnada.net> wrote:
On 11/15/26 18:31, Charlie Gibbs wrote:
Thanks, everyone, for your suggestions. Here's a history of my
various attempts.
<snipped>
Umm ... a two/three/four-stage CL 'solution' to
what SHOULD be easy ?
PDFs have a very useful place, but for SOME
purposes they're an over-complex disaster.
Perhaps, but non-experts are used to them and they're an
easy way to transmit images. And what I'm trying to do
is not easy; I've been handed a PDF file scanned at 2400
dpi in 24-bit colour, which I want to transform into a
monochrome file at 600 dpi. The 30-to-1 size advantage
turns a file transfer that requires Google Drive or Dropbox
into a simple e-mail attachment that displays quickly and
is more readable.
Yes, the ultimate solution would to have the person scanning
the file set up appropriate settings. But if that person
doesn't know the fine points of scanning and is at the mercy
of Apple (or Microsoft or whoever), this might not be an option.
Adding the capability to handle multi-page documents uncovered a
few problems. The sizes of the images seem to vary from page to
page, and some some pages have shadows that make it difficult to
make pencil marks visible without getting those big black blotches
I mentioned earlier. If I could persuade the person doing the
scanning to be careful to avoid shadows, my final script should
be able to do a wonderful job of cleaning up such files. This
might not be an option, though - most people seem unable or
unwilling to take the necessary care. At least the images
aren't keystoned...
Thanks, everyone, for your suggestions. Here's a history of my
various attempts. To keep things simple, I extracted the first
page of the 11-page original file (32,468,435 bytes) as follows:
$ pdftk hugefile.pdf cat 1 output testfile.pdf
This resulted in a file of 1,520,696 bytes; the reduction in size is
a lot more than 11 to 1, so pdftk must be doing some magic of its own.
$ pdfimages -list testfile.pdf
page num type width height color comp bpc enc interp object ID x-ppi y-ppi size ratio
--------------------------------------------------------------------------------------------
1 0 image 1928 2754 icc 3 8 jpeg yes 6 0 72 72 1481K 9.5%
This indicates colour (icc) with three 8-bit components per pixel.
$ identify testfile.pdf
testfile.pdf PDF 1944x2772 1944x2772+0+0 16-bit sRGB 9615B 0.010u 0:00.000
Now let's have GhostScript convert the file to grayscale,
using a suggestion from Nuno Silva:
$ gs -sDEVICE=pdfwrite -sColorConversionStrategy=Gray \
-sProcessColorModel=DeviceGray -sOutputFile=testfile2.pdf testfile.pdf
This converted the file to grayscale (8 bits per pixel, still with
JPEG compression; its size is now 359,415 bytes.
Following the URL given by Lawrence D'Oliveiro <https://ghostscript.readthedocs.io/gs10.08.0/Devices.html>
gave a bewildering array of possibilities (the GhostScript
command "devicenames ==" is even more intimidating).
Changing the device from pdfwrite to pngmono got the size down
to 112,386 bytes. The ImageMagick utility "identify" says it's
8-bit grayscale.
We're getting the file size down to something more reasonable.
However, contrast isn't all that good. Let's see whether we
can go for monochrome.
Working with Rich's suggestions, I ran the following:
$ gs -sDEVICE=pgmraw -sColorConversionStrategy=Gray \
-sProcessColorModel=DeviceGray -sOutputFile=testfile3.pgm testfile.pdf
$ pgmtopbm -threshold -value 0.82 testfile3.pgm | pnmtotiff -g4 >testfile3.tif
$ tiff2pdf -o testfile3.pdf testfile3.tif
Bingo! I wound up with a crisp PDF file whose size is only 48,071
bytes: a 30-to-1 reduction. This size is typical of what I get
when I do my own scanning. And by tweaking the threshold value
from 0.82 to 0.9 I was able to bring out penciled notes without
getting black blotches everywhere.
Adding the capability to handle multi-page documents uncovered a
few problems. The sizes of the images seem to vary from page to
page, and some some pages have shadows that make it difficult to
make pencil marks visible without getting those big black blotches
I mentioned earlier. If I could persuade the person doing the
scanning to be careful to avoid shadows, my final script should
be able to do a wonderful job of cleaning up such files. This
might not be an option, though - most people seem unable or
unwilling to take the necessary care. At least the images
aren't keystoned...
Thanks, everyone, for your help!
(And if you end up converting the document to monochrome, it's probably
worth considering at least comparing with DjVu unless you need it to be
PDF.)
Consider having a look at ImageMagick's convert with -lat, "local
adaptive thresholding". Sadly, my notes don't go beyond saying it's
better than just threshold... example command line:
convert in.pnm -lat 100x700-15% out.pbm
<https://usage.imagemagick.org/transform/#lat>
(unpaper might also be useful, depending on how the input documents
are?:)
<https://flameeyes.com/projects/unpaper>
(And if you end up converting the document to monochrome, it's probably
worth considering at least comparing with DjVu unless you need it to be
PDF.)
GIMP might possibly help? Off the top of my head, import your pdf with
pages as layers, fiddle with colour, size and resolution, then export
layers as pages in a pdf.
On 2026-11-16, Mike Scott <usenet.16@scottsonline.org.uk.invalid> wrote:
GIMP might possibly help? Off the top of my head, import your pdf with
pages as layers, fiddle with colour, size and resolution, then export
layers as pages in a pdf.
That's an interesting possibility. And maybe the Gimp can do some
equivalent of dodging to try to lighten up the shawdowed areas
before converting to monochrome.
Yes, the ultimate solution would to have the person scanning the file
set up appropriate settings. But if that person doesn't know the
fine points of scanning and is at the mercy of Apple (or Microsoft or whoever), this might not be an option.
On 2026-11-16 09:56, Nuno Silva wrote:
(And if you end up converting the document to monochrome, it's probably
worth considering at least comparing with DjVu unless you need it to be
PDF.)
How do you handle conversion to djvu in Linux? I have found few
people that use this format.
"Carlos E.R." wrote:
How do you handle conversion to djvu in Linux? I have found few people
that use this format.
I can see that at one time, DjVu would have had advantages over Pdf, is
that still true, especially with Pdf/A ?
I'm dealing with musicians here, not computer geeks. Going to some
obscure new format would be just one more headache to deal with.
Charlie Gibbs <cgibbs@kltpzyxm.invalid> wrote:
I'm dealing with musicians here, not computer geeks. Going to some
obscure new format would be just one more headache to deal with.
Then don't bother with anything other than PDF. That one they will be
able to open and view. Anything else, yes, a headache for you.
Indeed. Thanks for the help. Unfortunately, my tests have suddenly
stopped working
On 2026-11-16, Nuno Silva <nunojsilva@invalid.invalid> wrote:
Consider having a look at ImageMagick's convert with -lat, "local
adaptive thresholding". Sadly, my notes don't go beyond saying it's
better than just threshold... example command line:
convert in.pnm -lat 100x700-15% out.pbm
<https://usage.imagemagick.org/transform/#lat>
That sounds interesting! I'll give it a look.
(unpaper might also be useful, depending on how the input documents
are?:)
<https://flameeyes.com/projects/unpaper>
On 2026-11-16, Rich <rich@example.invalid> wrote:
Charlie Gibbs <cgibbs@kltpzyxm.invalid> wrote:
I'm dealing with musicians here, not computer geeks. Going to some
obscure new format would be just one more headache to deal with.
Then don't bother with anything other than PDF. That one they will be
able to open and view. Anything else, yes, a headache for you.
Indeed. Thanks for the help. Unfortunately, my tests have suddenly
stopped working - pgmtopbm has seemingly started ignoring my threshold setting. Stripping down the commands to work on a single-page PDF:
$ gs -dNOPAUSE -q -r300 -sDEVICE=pgmraw -dBATCH -sOutputFile=testfile3.pgm testfile.pdf
$ pgmtopbm -threshold -value 0.82 testfile3.pgm >testfile3.pnm
$ pnmtotiff -g4 <testfile3.pnm >testfile3.tif
Is there something wrong with my pgmtopbm command?
If I remove the "-threshold -value 0.82" parameters
the output (testfile3.pnm) is exactly the same.
Charlie Gibbs <cgibbs@kltpzyxm.invalid> wrote:
Indeed. Thanks for the help. Unfortunately, my tests have suddenly
stopped working - pgmtopbm has seemingly started ignoring my threshold
setting. Stripping down the commands to work on a single-page PDF:
$ gs -dNOPAUSE -q -r300 -sDEVICE=pgmraw -dBATCH -sOutputFile=testfile3.pgm testfile.pdf
$ pgmtopbm -threshold -value 0.82 testfile3.pgm >testfile3.pnm
$ pnmtotiff -g4 <testfile3.pnm >testfile3.tif
Is there something wrong with my pgmtopbm command?
Not that I see from the above example, other than the result file
should be named ".pbm" as it is a Portable Bit Map (monochrome) file.
But so long as you use the same name to feed into pnmtotiff it won't
matter a bit what you name it.
If I remove the "-threshold -value 0.82" parameters
the output (testfile3.pnm) is exactly the same.
Weird. I would expect some change, as it is supposed to be outputting
pbm, and you should be sending in greyscale, so the two files should
differ.
"Carlos E.R." wrote:
How do you handle conversion to djvu in Linux? I have found few people
that use this format.
I can see that at one time, DjVu would have had advantages over Pdf, is
that still true, especially with Pdf/A ?
Charlie Gibbs <cgibbs@kltpzyxm.invalid> wrote:
Yes, the ultimate solution would to have the person scanning the file
set up appropriate settings. But if that person doesn't know the
fine points of scanning and is at the mercy of Apple (or Microsoft or
whoever), this might not be an option.
Yes, the "proper fix" here is to have whomever scanned the original
paper do a proper scan from the beginning.
Sadly, most normies have no idea and just accept the defaults, and then
the scanner software makers further confuse things by doing silly
things like making the UI entry for "black and white" actually map to
8-bit grey scale because most folks think of "black and white photos"
when they think black & white for a scan, which itself "is" a greyscale image. Actually getting a monochrome scan out of some of the software
(esp. the bundled software) can be a royal pain.
Charlie Gibbs <cgibbs@kltpzyxm.invalid> wrote:
Yes, the ultimate solution would to have the person scanning the file
set up appropriate settings. But if that person doesn't know the
fine points of scanning and is at the mercy of Apple (or Microsoft or
whoever), this might not be an option.
Yes, the "proper fix" here is to have whomever scanned the original
paper do a proper scan from the beginning.
Sadly, most normies have no idea and just accept the defaults, and then
the scanner software makers further confuse things by doing silly
things like making the UI entry for "black and white" actually map to
8-bit grey scale because most folks think of "black and white photos"
when they think black & white for a scan, which itself "is" a greyscale image. Actually getting a monochrome scan out of some of the software
(esp. the bundled software) can be a royal pain.
| Sysop: | Jacob Catayoc |
|---|---|
| Location: | Pasay City, Metro Manila, Philippines |
| Users: | 5 |
| Nodes: | 4 (0 / 4) |
| Uptime: | 498677:27:08 |
| Calls: | 212 |
| Calls today: | 3 |
| Files: | 910 |
| D/L today: |
31 files (22,593K bytes) |
| Messages: | 69,826 |