pg_basebackup fails with long tablespace paths

[Tom Lane]

My Salesforce colleague Thomas Fanghaenel observed that the TAP tests
for pg_basebackup fail when run in a sufficiently deeply-nested directory
tree. The cause appears to be that we rely on standard "tar" format
to represent the symlink for a tablespace, and POSIX tar format has a
hard-wired restriction of 99 bytes in a symlink's expansion.

What do we want to do about this? I think a minimum expectation would be
for pg_basebackup to notice and complain when it's trying to create an
unworkably long symlink entry, but it would be far better if we found a
way to cope instead.

One thing we could possibly do without reinventing "tar" is to avoid using
absolute path names if a PGDATA-relative one would do.

regards, tom lane

[Peter Eisentraut]

Post by Tom Lane
What do we want to do about this? I think a minimum expectation would be
for pg_basebackup to notice and complain when it's trying to create an
unworkably long symlink entry, but it would be far better if we found a
way to cope instead.

Isn't it the backend that should error out before sending truncated
files names?

src/port/tar.c:

/* Name 100 */
sprintf(&h[0], "%.99s", filename);

And then do we need to prevent the creation of tablespaces that can't be
backed up?

Post by Tom Lane
One thing we could possibly do without reinventing "tar" is to avoid >

using

Post by Tom Lane
absolute path names if a PGDATA-relative one would do.

Maybe we could hack up the tar format to store the symlink target as the
file body, like cpio does. Of course then we'd lose the property of
this actually being tar.

[Amit Kapila]

Post by Tom Lane
My Salesforce colleague Thomas Fanghaenel observed that the TAP tests
for pg_basebackup fail when run in a sufficiently deeply-nested directory
tree. The cause appears to be that we rely on standard "tar" format
to represent the symlink for a tablespace, and POSIX tar format has a
hard-wired restriction of 99 bytes in a symlink's expansion.
What do we want to do about this? I think a minimum expectation would be
for pg_basebackup to notice and complain when it's trying to create an
unworkably long symlink entry, but it would be far better if we found a
way to cope instead.

One way to cope with such a situation could be that during backup we create
a backup symlink file which contains listing of symlinks and then archive
recovery recreates it. Basically this is the solution (patch), I have
proposed
for Windows [1].


[1] - https://commitfest.postgresql.org/action/patch_view?id=1512

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com