bug#12130: "sudo make install" applies umask to new directories

[Jason Eisner]

Hi automake folks,

There are many, many generic and specific installation guides that tell you
to type something like
./configure && make && sudo make install
and I've been typing that for years.

Unfortunately, when installing OpenFST, I just discovered that the "sudo
make install" part doesn't always work. My umask is 0077, so any
directories created with sudo (hence owned by root) are unreadable to
ordinary users, including myself.

Since following the documentation yields mysterious error messages and
hard-to-fix errors[*] rather than the intended behavior, I believe that you
should make ONE of the following changes:

1. Fix the documentation: Correct the instructions to users to warn them
that they should reset their umask before typing sudo make install.

2. Fix the behavior: Have "make install" set the permissions on
directories, just as it already does on files. For example, by using -m
755 as an option to mkdir, or otherwise overriding the umask.

3. Don't fix the behavior outright, if you think for some reason that the
user's umask should sometimes be respected on new directories. But then at
least have "make install" issue a warning or a y/n prompt when the current
umask (or for that matter, restrictive permissions on existing directories)
would result in installed resources that are not publicly readable.

My own preference would be for 2. I can't understand why make install
treats directory permissions inconsistently with file permissions. And I
can't see why a single-user preference like a umask should be reflected in
a global installation.

However, if there is some reason not to do 2., then I think you should do
1. or 3. to save at least some of the headaches.[*]

This problem doesn't affect all packages -- only installations that create
new directories. But it has been raised repeatedly over the years. To
find numerous past reports, just do websearches such as
umask automake
umask directories "make install"
However, I think it has usually been raised with maintainers of downstream
packages, who are not equipped to fix it. Nothing comes up when I search
for umask on http://lists.gnu.org/archive/html/bug-automake/ .

Thanks a lot for your attention!

-jason

[*] Following the standard installation instructions leads to a silent
failure of installation. The cockeyed installation can result in
mysterious error messages when you try to use it. E.g., I got messages
saying that the command-line utilities couldn't find their .so files.
Fixing the mistaken permissions is also tricky. Even if you figure out
that your umask was at fault, you can't just change your umask and run
"sudo make install" again. That's because the directories already exist
and rerunning "mkdir -p" on them is a no-op. So you have to figure out
which directories were created and chmod them manually. I hope this makes
clear why having some kind of fix is important ...

[Jason Eisner]

Followup -- see this thread from 2004:
http://www.digipedia.pl/usenet/thread/16496/9413/

Looks like at one point, someone fixed this problem in the way I
suggested. But then a user on that thread found directory permissions of
755 too restrictive, which may have gotten the fix removed.

One way to make everyone happy might be to ensure that a newly created
directory has permissions of *at least* 755. (Although it's a bit peculiar
to handle directories this way when files are standardly 644 ...)

-jason

[Stefano Lattarini]

Post by Jason Eisner
Hi automake folks,

Hi Jason, thanks for the report.

Post by Jason Eisner
There are many, many generic and specific installation guides that tell you
to type something like
./configure && make && sudo make install
and I've been typing that for years.
Unfortunately, when installing OpenFST, I just discovered that the "sudo
make install" part doesn't always work. My umask is 0077, so any
directories created with sudo (hence owned by root) are unreadable to
ordinary users, including myself.

This is indeed annoying.

Post by Jason Eisner
Since following the documentation yields mysterious error messages and
hard-to-fix errors[*] rather than the intended behavior, I believe that you
1. Fix the documentation: Correct the instructions to users to warn them
that they should reset their umask before typing sudo make install.

This sounds like a good idea. Care to attempt a patch? Otherwise, I'll
get to write it myself eventually (not right now).

But then, reading further ...

Post by Jason Eisner
2. Fix the behavior: Have "make install" set the permissions on
directories, just as it already does on files. For example, by using -m
755 as an option to mkdir, or otherwise overriding the umask.

... I see that a patch in this direction had already been proposed:
<http://lists.gnu.org/archive/html/bug-automake/2003-05/msg00011.html>
but apparently never applied (the discussion died out). Maybe we could
just resurrect it ...

Any opinion from the other autotoolers?

Post by Jason Eisner
3. Don't fix the behavior outright, if you think for some reason that the
user's umask should sometimes be respected on new directories. But then at
least have "make install" issue a warning

That would likely be lost in the very verbose "make install" output,
sadly.

Post by Jason Eisner
or a y/n prompt when the current umask

Oh no, our rules are not going to get interactive by default; not even
conditionally ("if input/output is attached to a terminal, then print
a prompt and ..."). There lies madness.

Post by Jason Eisner
(or for that matter, restrictive permissions on existing directories)
would result in installed resources that are not publicly readable.
My own preference would be for 2.

I'm undecided among 1 and 2, but leaning toward 2 by now. I'll wait a
few days for some feedback by the other autotoolers before proceeding.

Post by Jason Eisner
I can't understand why make install
treats directory permissions inconsistently with file permissions.

I'd guess hysterical raisins -- as is the case with many other warts and
rough edges in Automake.

Post by Jason Eisner
And I can't see why a single-user preference like a umask should be
reflected in a global installation.

I agree it shouldn't.

Post by Jason Eisner
However, if there is some reason not to do 2., then I think you should do
1. or 3. to save at least some of the headaches.[*]
This problem doesn't affect all packages -- only installations that create
new directories. But it has been raised repeatedly over the years. To
find numerous past reports, just do websearches such as
umask automake

Thanks for this suggestion. It pointed me to the older patch I referenced
above.

Post by Jason Eisner
umask directories "make install"
However, I think it has usually been raised with maintainers of downstream
packages, who are not equipped to fix it. Nothing comes up when I search
for umask on http://lists.gnu.org/archive/html/bug-automake/ .
Thanks a lot for your attention!
-jason
[*] Following the standard installation instructions leads to a silent
failure of installation. The cockeyed installation can result in
mysterious error messages when you try to use it. E.g., I got messages
saying that the command-line utilities couldn't find their .so files.
Fixing the mistaken permissions is also tricky. Even if you figure out
that your umask was at fault, you can't just change your umask and run
"sudo make install" again. That's because the directories already exist
and rerunning "mkdir -p" on them is a no-op. So you have to figure out
which directories were created and chmod them manually. I hope this makes
clear why having some kind of fix is important ...

Thanks,
Stefano

[Jason Eisner]

Dear Stefano - Thanks very much for your sympathetic reply on the directory
permissions issue. As we haven't heard back further after a few days, I'd
suggest going forward with plan 2.

You found that plan 2 was formerly proposed at <
http://lists.gnu.org/archive/html/bug-automake/2003-05/msg00011.html>. You
suspected that it wasn't applied. I believe that actually, it was applied,
but then reverted because it broke someone's build. As I wrote in a

Post by Jason Eisner
http://www.digipedia.pl/usenet/thread/16496/9413/
Looks like at one point, someone fixed this problem in the way I
suggested. But then a user on that thread found directory permissions of
755 too restrictive, which may have gotten the fix removed.
One way to make everyone happy might be to ensure that a newly created
directory has permissions of *at least* 755. (Although it's a bit peculiar
to handle directories this way when files are standardly 644 ...)

In other words, perhaps the mode of the newly created directory (the XXX in
mkdir -p -m XXX) needs to be be the bitwise "OR" of 755 and the current
umask.

By erring on the side of permissiveness, this hack allows "make install" to
succeed from any user account (as I hoped), while still preserving backward
compatibility (for the guy who complained above in 2004 that he wanted it
to work differently from HIS user account).

This is one of those backward compatibility hacks that results in inelegant
behavior. I wish the 2004 guy had not relied on getting umask permissions
for directories (he couldn't have relied on umask permissions for files,
since those are standardly 644). But it might be worth it if anyone is
still relying on this behavior. If so, maybe the generated makefile should
have a comment explaining that the "OR" with umask is only for backward
compatibility with previous versions of automake.

cheers, jason

[Stefano Lattarini]

Post by Jason Eisner
Dear Stefano - Thanks very much for your sympathetic reply on the directory
permissions issue. As we haven't heard back further after a few days, I'd
suggest going forward with plan 2.
You found that plan 2 was formerly proposed at
<http://lists.gnu.org/archive/html/bug-automake/2003-05/msg00011.html>. You
suspected that it wasn't applied. I believe that actually, it was applied,
but then reverted because it broke someone's build.

Post by Jason Eisner
http://www.digipedia.pl/usenet/thread/16496/9413/

You're right; here's the same discussion archived on a more "official" site:

<http://lists.gnu.org/archive/html/automake/2004-01/msg00013.html>

Thanks for correcting me and saving me from a potential mistake!

Post by Jason Eisner

Post by Jason Eisner
Looks like at one point, someone fixed this problem in the way I
suggested. But then a user on that thread found directory permissions of
755 too restrictive, which may have gotten the fix removed.
One way to make everyone happy might be to ensure that a newly created
directory has permissions of *at least* 755.
(Although it's a bit peculiar to handle directories this way when
files are standardly 644 ...)

Exactly. I believe we should either re-install the patch forcing the 755
permission on the intermediate directories, or enhance the install rules
to respect the user umask for the installation of regular files as well.
This is something to consider carefully though, and only to be changed (if
we decide to change it) in the next major automake version. For now, we
should content ourselves with the documentation fix you proposed.

Post by Jason Eisner
In other words, perhaps the mode of the newly created directory (the XXX in
mkdir -p -m XXX) needs to be be the bitwise "OR" of 755 and the current
umask.

I disagree. If we want to honour the user umask, we should do so fully;
half-hearted solutions (in one direction or another) will end up creating
confusion, and more problems than they solve.

Post by Jason Eisner
By erring on the side of permissiveness, this hack allows "make install" to
succeed from any user account (as I hoped), while still preserving backward
compatibility (for the guy who complained above in 2004 that he wanted it
to work differently from HIS user account).
This is one of those backward compatibility hacks that results in inelegant
behavior. I wish the 2004 guy had not relied on getting umask permissions
for directories (he couldn't have relied on umask permissions for files,
since those are standardly 644). But it might be worth it if anyone is
still relying on this behavior.
If so, maybe the generated makefile should
have a comment explaining that the "OR" with umask is only for backward
compatibility with previous versions of automake.
cheers, jason

Thanks,
Stefano

[Jason Eisner]

On Fri, Aug 10, 2012 at 5:47 AM, Stefano Lattarini <

Post by Stefano Lattarini
This is something to consider carefully though, and only to be changed (if
we decide to change it) in the next major automake version. For now, we
should content ourselves with the documentation fix you proposed.

When is the next major version? I'm guessing that a mere docfix won't help
existing packages because they are unlikely to replace their documentation
when they rerun automake (?).

Post by Stefano Lattarini
In other words, perhaps the mode of the newly created directory (the XXX in

Post by Jason Eisner
mkdir -p -m XXX) needs to be be the bitwise "OR" of 755 and the current
umask.

I disagree. If we want to honour the user umask, we should do so fully;
half-hearted solutions (in one direction or another) will end up creating
confusion, and more problems than they solve.

Perhaps. But if you wish to preserve backward compatibility, that appears
to require some degree of inconsistency (as is typical). In particular, it
will require treating umask differently on files and directories, because
that is what has been done in the past. Making the behavior consistent
seems likely to break backward compatibility. If we start honoring umask
on everything, that will break a large number of existing installation
habits or scripts that relied on install to use 644 on files. If we start
ignoring umask on everything, that will break a smaller number of existing
installation habits or scripts that relied on a more permissive umask on
directories (i.e., the guy who complained in 2004).

The only ways I can see to make it consistent between files and directories
are to use the "OR" solution for both, or to ignore umask by default but
have a command-line flag on "sudo make install" that says "please respect
my umask" (for the few users who want to rely on it to get nonstandard
installation permissions at their site).

[Jason Eisner]

Post by Jason Eisner
The only ways I can see to make it consistent between files and
directories are to use the "OR" solution for both, or to ignore umask by
default but have a command-line flag on "sudo make install" that says
"please respect my umask" (for the few users who want to rely on it to get
nonstandard installation permissions at their site).

Better idea:

Have default 644 for files and 755 for directories, but let the user
override this by explicitly specifying any of
* the desired file permissions
* the desired directory permissions for newly created directories
* the desired group owner for newly created files and directories
via command-line options on "make install".

The 644/755 default is consistent and will work right for almost everyone.
But the command-line flag is useful for users who want to install with
different conventions. I like this better than a 1-bit flag that says
"please respect my umask."

-jason

[Peter Johansson]

Post by Jason Eisner
Have default 644 for files and 755 for directories, but let the user
override this by explicitly specifying any of
* the desired file permissions
* the desired directory permissions for newly created directories
* the desired group owner for newly created files and directories
via command-line options on "make install".

You can already do this. You can, e.g., install packages with

make install MKDIR_P="mkdir -p -m 700"

Cheers,
Peter

[Jason Eisner]

Post by Peter Johansson

Post by Jason Eisner
Have default 644 for files and 755 for directories, but let the user
override this by explicitly specifying any of
* the desired file permissions
* the desired directory permissions for newly created directories
* the desired group owner for newly created files and directories
via command-line options on "make install".
You can already do this. You can, e.g., install packages with

make install MKDIR_P="mkdir -p -m 700"

Good! So then, why not correct the default for new directories to 755, as
originally proposed?

cheers, jason

[didi]

Post by Peter Johansson

Post by Jason Eisner
Have default 644 for files and 755 for directories, but let the user
override this by explicitly specifying any of
* the desired file permissions
* the desired directory permissions for newly created directories
* the desired group owner for newly created files and directories
via command-line options on "make install".

You can already do this. You can, e.g., install packages with
make install MKDIR_P="mkdir -p -m 700"

Unfortunately this doesn't seem to work properly, as the parent directories
still retain the permissions of the user.

$ mkdir -p -m 755 foo/bar

drwx------ foo/
drwxr-xr-x bar/

[Peter Johansson]

[CC coreutils this is
http://lists.gnu.org/archive/html/bug-automake/2012-08/msg00001.html]

Post by didi

Post by Peter Johansson
You can already do this. You can, e.g., install packages with

Post by Peter Johansson
make install MKDIR_P="mkdir -p -m 700"

Unfortunately this doesn't seem to work properly, as the parent directories
still retain the permissions of the user.
$ mkdir -p -m 755 foo/bar
drwx------ foo/
drwxr-xr-x bar/

Hi Didi,

That was unexpected and unfortunate IMVHO. I see the same behaviour on
my local system, in other words,

mkdir -p -m 700 /tmp/foo/bar

creates bar with permissions 700 and bar with 776 (my umask). I wonder
is that behaviour is mandated by some standard or if there is room for
improvement in coreutils here (CC:ed).

Thanks,
Peter

[Eric Blake]

tag 12130 notabug
thanks

Post by Peter Johansson
mkdir -p -m 700 /tmp/foo/bar
creates bar with permissions 700 and bar with 776 (my umask). I wonder
is that behaviour is mandated by some standard

Yes - POSIX requires this:

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/mkdir.html

-p
Create any missing intermediate pathname components.

For each dir operand that does not name an existing directory,
effects equivalent to those caused by the following command shall occur:

mkdir -p -m $(umask -S),u+wx $(dirname dir) &&
mkdir [-m mode] dir

where the -m mode option represents that option supplied to the
original invocation of mkdir, if any.

Post by Peter Johansson
or if there is room for
improvement in coreutils here (CC:ed).

Since POSIX requires the existing behavior, there's unfortunately no
room for improvement, and I'm closing this as not a bug in coreutils.