Postgresql 10 pg_basebackup not backing up pg_control file
up vote
0
down vote
favorite
I'm new to administrating postgresql 10 database so please bear with me (I'm from Oracle background). I was hoping a kind soul could point me to the right direction with our current problem.
We use pg_base to backup our current production system and for some reason since a couple of days ago pg_base has stopped backing up the database's pg_control file as well as postgres.conf which ofcourse renders the backup useless for restoration.
Below is the command that we use to initiate the backup
/usr/bin/docker exec -i --user postgres asf-db1 pg_basebackup --wal-method=fetch -D /var/lib/postgresql/data/pgdata/master_backup/latest -P -v --format=tar
The logs does not indicate anything suspicious and the issue only became apparent when I did a test restore.
Below is an extract of the backup logs
pg_basebackup: initiating base backup, waiting for checkpoint to complete
pg_basebackup: checkpoint completed
pg_basebackup: write-ahead log start point: 6/BF000178 on timeline 1
0/1519047 kB (0%), 0/1 tablespace (...ta/master_backup/latest/base.tar)
397408/1519047 kB (26%), 0/1 tablespace (...ta/master_backup/latest/base.tar
)^M1019400/1519047 kB (67%), 0/1 tablespace (...ta/master_backup/latest/base.
tar)^M1565927/1565927 kB (100%), 0/1 tablespace (...ta/master_backup/latest/b
t/base.tar)^M2164615/2164615 kB (100%), 0/1 tablespace (...ta/master_backup/l
p/latest/base.tar)^M2751015/2751015 kB (100%), 0/1 tablespace (...ta/master_b
r_backup/latest/base.tar)^M3061585/3061585 kB (100%), 0/1 tablespace (...ta/m
pg_basebackup: write-ahead log end point: 6/C0000050
pg_basebackup: base backup completed
Backup completed at: Wed 14 Nov 18:00:32 GMT 2018
postgresql
asked Nov 15 at 16:53
Mr.Eva
103
add a comment |
up vote
0
down vote
favorite
I'm new to administrating postgresql 10 database so please bear with me (I'm from Oracle background). I was hoping a kind soul could point me to the right direction with our current problem.
We use pg_base to backup our current production system and for some reason since a couple of days ago pg_base has stopped backing up the database's pg_control file as well as postgres.conf which ofcourse renders the backup useless for restoration.
Below is the command that we use to initiate the backup
/usr/bin/docker exec -i --user postgres asf-db1 pg_basebackup --wal-method=fetch -D /var/lib/postgresql/data/pgdata/master_backup/latest -P -v --format=tar
The logs does not indicate anything suspicious and the issue only became apparent when I did a test restore.
Below is an extract of the backup logs
pg_basebackup: initiating base backup, waiting for checkpoint to complete
pg_basebackup: checkpoint completed
pg_basebackup: write-ahead log start point: 6/BF000178 on timeline 1
0/1519047 kB (0%), 0/1 tablespace (...ta/master_backup/latest/base.tar)
397408/1519047 kB (26%), 0/1 tablespace (...ta/master_backup/latest/base.tar
)^M1019400/1519047 kB (67%), 0/1 tablespace (...ta/master_backup/latest/base.
tar)^M1565927/1565927 kB (100%), 0/1 tablespace (...ta/master_backup/latest/b
t/base.tar)^M2164615/2164615 kB (100%), 0/1 tablespace (...ta/master_backup/l
p/latest/base.tar)^M2751015/2751015 kB (100%), 0/1 tablespace (...ta/master_b
r_backup/latest/base.tar)^M3061585/3061585 kB (100%), 0/1 tablespace (...ta/m
pg_basebackup: write-ahead log end point: 6/C0000050
pg_basebackup: base backup completed
Backup completed at: Wed 14 Nov 18:00:32 GMT 2018
postgresql
asked Nov 15 at 16:53
Mr.Eva
103
That is weird. Permissions on the files are OK?pg_basebackupwill always sendpg_controllast, so could it be that you are running out of space and the file is incomplete? That would cause an error though.
– Laurenz Albe
Nov 15 at 17:11
yes both files are owned by postgres and space is not a problem either and the rest of the datafiles and wal files can be found in the backup.
– Mr.Eva
Nov 15 at 17:23
Hard to debug without examining your system. One hard core way to debug this is tostracethe database walsender process that handles the backup and see what's going on.
– Laurenz Albe
Nov 15 at 17:31
Thanks for your help guys. The issue is fixed now it actually originated from the fact that the backup script was cronned multiple times in parallel. It was from an automated salt configuration.
– Mr.Eva
Nov 16 at 15:15
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I'm new to administrating postgresql 10 database so please bear with me (I'm from Oracle background). I was hoping a kind soul could point me to the right direction with our current problem.
We use pg_base to backup our current production system and for some reason since a couple of days ago pg_base has stopped backing up the database's pg_control file as well as postgres.conf which ofcourse renders the backup useless for restoration.
Below is the command that we use to initiate the backup
/usr/bin/docker exec -i --user postgres asf-db1 pg_basebackup --wal-method=fetch -D /var/lib/postgresql/data/pgdata/master_backup/latest -P -v --format=tar
The logs does not indicate anything suspicious and the issue only became apparent when I did a test restore.
Below is an extract of the backup logs
pg_basebackup: initiating base backup, waiting for checkpoint to complete
pg_basebackup: checkpoint completed
pg_basebackup: write-ahead log start point: 6/BF000178 on timeline 1
0/1519047 kB (0%), 0/1 tablespace (...ta/master_backup/latest/base.tar)
397408/1519047 kB (26%), 0/1 tablespace (...ta/master_backup/latest/base.tar
)^M1019400/1519047 kB (67%), 0/1 tablespace (...ta/master_backup/latest/base.
tar)^M1565927/1565927 kB (100%), 0/1 tablespace (...ta/master_backup/latest/b
t/base.tar)^M2164615/2164615 kB (100%), 0/1 tablespace (...ta/master_backup/l
p/latest/base.tar)^M2751015/2751015 kB (100%), 0/1 tablespace (...ta/master_b
r_backup/latest/base.tar)^M3061585/3061585 kB (100%), 0/1 tablespace (...ta/m
pg_basebackup: write-ahead log end point: 6/C0000050
pg_basebackup: base backup completed
Backup completed at: Wed 14 Nov 18:00:32 GMT 2018
postgresql
asked Nov 15 at 16:53
Mr.Eva
103
I'm new to administrating postgresql 10 database so please bear with me (I'm from Oracle background). I was hoping a kind soul could point me to the right direction with our current problem.
We use pg_base to backup our current production system and for some reason since a couple of days ago pg_base has stopped backing up the database's pg_control file as well as postgres.conf which ofcourse renders the backup useless for restoration.
Below is the command that we use to initiate the backup
/usr/bin/docker exec -i --user postgres asf-db1 pg_basebackup --wal-method=fetch -D /var/lib/postgresql/data/pgdata/master_backup/latest -P -v --format=tar
The logs does not indicate anything suspicious and the issue only became apparent when I did a test restore.
Below is an extract of the backup logs
pg_basebackup: initiating base backup, waiting for checkpoint to complete
pg_basebackup: checkpoint completed
pg_basebackup: write-ahead log start point: 6/BF000178 on timeline 1
0/1519047 kB (0%), 0/1 tablespace (...ta/master_backup/latest/base.tar)
397408/1519047 kB (26%), 0/1 tablespace (...ta/master_backup/latest/base.tar
)^M1019400/1519047 kB (67%), 0/1 tablespace (...ta/master_backup/latest/base.
tar)^M1565927/1565927 kB (100%), 0/1 tablespace (...ta/master_backup/latest/b
t/base.tar)^M2164615/2164615 kB (100%), 0/1 tablespace (...ta/master_backup/l
p/latest/base.tar)^M2751015/2751015 kB (100%), 0/1 tablespace (...ta/master_b
r_backup/latest/base.tar)^M3061585/3061585 kB (100%), 0/1 tablespace (...ta/m
pg_basebackup: write-ahead log end point: 6/C0000050
pg_basebackup: base backup completed
Backup completed at: Wed 14 Nov 18:00:32 GMT 2018
postgresql
postgresql
asked Nov 15 at 16:53
Mr.Eva
103
asked Nov 15 at 16:53
Mr.Eva
103
edited Nov 15 at 17:27
asked Nov 15 at 16:53
Mr.Eva
103
asked Nov 15 at 16:53
Mr.Eva
103
asked Nov 15 at 16:53
Mr.Eva
103
103
That is weird. Permissions on the files are OK?pg_basebackupwill always sendpg_controllast, so could it be that you are running out of space and the file is incomplete? That would cause an error though.
– Laurenz Albe
Nov 15 at 17:11
yes both files are owned by postgres and space is not a problem either and the rest of the datafiles and wal files can be found in the backup.
– Mr.Eva
Nov 15 at 17:23
Hard to debug without examining your system. One hard core way to debug this is tostracethe database walsender process that handles the backup and see what's going on.
– Laurenz Albe
Nov 15 at 17:31
Thanks for your help guys. The issue is fixed now it actually originated from the fact that the backup script was cronned multiple times in parallel. It was from an automated salt configuration.
– Mr.Eva
Nov 16 at 15:15
add a comment |
That is weird. Permissions on the files are OK?pg_basebackupwill always sendpg_controllast, so could it be that you are running out of space and the file is incomplete? That would cause an error though.
– Laurenz Albe
Nov 15 at 17:11
yes both files are owned by postgres and space is not a problem either and the rest of the datafiles and wal files can be found in the backup.
– Mr.Eva
Nov 15 at 17:23
Hard to debug without examining your system. One hard core way to debug this is tostracethe database walsender process that handles the backup and see what's going on.
– Laurenz Albe
Nov 15 at 17:31
Thanks for your help guys. The issue is fixed now it actually originated from the fact that the backup script was cronned multiple times in parallel. It was from an automated salt configuration.
– Mr.Eva
Nov 16 at 15:15
That is weird. Permissions on the files are OK?
pg_basebackup will always send pg_control last, so could it be that you are running out of space and the file is incomplete? That would cause an error though.– Laurenz Albe
Nov 15 at 17:11
That is weird. Permissions on the files are OK?
pg_basebackup will always send pg_control last, so could it be that you are running out of space and the file is incomplete? That would cause an error though.– Laurenz Albe
Nov 15 at 17:11
yes both files are owned by postgres and space is not a problem either and the rest of the datafiles and wal files can be found in the backup.
– Mr.Eva
Nov 15 at 17:23
yes both files are owned by postgres and space is not a problem either and the rest of the datafiles and wal files can be found in the backup.
– Mr.Eva
Nov 15 at 17:23
Hard to debug without examining your system. One hard core way to debug this is to
strace the database walsender process that handles the backup and see what's going on.– Laurenz Albe
Nov 15 at 17:31
Hard to debug without examining your system. One hard core way to debug this is to
strace the database walsender process that handles the backup and see what's going on.– Laurenz Albe
Nov 15 at 17:31
Thanks for your help guys. The issue is fixed now it actually originated from the fact that the backup script was cronned multiple times in parallel. It was from an automated salt configuration.
– Mr.Eva
Nov 16 at 15:15
Thanks for your help guys. The issue is fixed now it actually originated from the fact that the backup script was cronned multiple times in parallel. It was from an automated salt configuration.
– Mr.Eva
Nov 16 at 15:15
add a comment |
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53324327%2fpostgresql-10-pg-basebackup-not-backing-up-pg-control-file%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53324327%2fpostgresql-10-pg-basebackup-not-backing-up-pg-control-file%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
That is weird. Permissions on the files are OK?
pg_basebackupwill always sendpg_controllast, so could it be that you are running out of space and the file is incomplete? That would cause an error though.– Laurenz Albe
Nov 15 at 17:11
yes both files are owned by postgres and space is not a problem either and the rest of the datafiles and wal files can be found in the backup.
– Mr.Eva
Nov 15 at 17:23
Hard to debug without examining your system. One hard core way to debug this is to
stracethe database walsender process that handles the backup and see what's going on.– Laurenz Albe
Nov 15 at 17:31
Thanks for your help guys. The issue is fixed now it actually originated from the fact that the backup script was cronned multiple times in parallel. It was from an automated salt configuration.
– Mr.Eva
Nov 16 at 15:15