How does journald know the PID of a process that produces log data?
When I look at journalctl
, it tells me the PID and the program name(or service name?) of a log entry.
Then I wondered, logs are created by other processes, how do systemd-journald
know the PID of these processes when processes may only write raw strings to the unix domain socket which systemd-journald
is listenning. Also, do sytemd-journald
always use the same technique to detect the PID of a piece of log data even when processes are producing log using functions like sd_journal_sendv()
?
Is there any documentation I should read about this?
I read JdeBP's answer and know systemd-journald
listen on an Unix Domian Socket, but even if can know the peer socket address who send the log message, how does it know the PID? What if that sending socket is opened by many non-parent-children processes?
systemd-journald ipc
add a comment |
When I look at journalctl
, it tells me the PID and the program name(or service name?) of a log entry.
Then I wondered, logs are created by other processes, how do systemd-journald
know the PID of these processes when processes may only write raw strings to the unix domain socket which systemd-journald
is listenning. Also, do sytemd-journald
always use the same technique to detect the PID of a piece of log data even when processes are producing log using functions like sd_journal_sendv()
?
Is there any documentation I should read about this?
I read JdeBP's answer and know systemd-journald
listen on an Unix Domian Socket, but even if can know the peer socket address who send the log message, how does it know the PID? What if that sending socket is opened by many non-parent-children processes?
systemd-journald ipc
See: freedesktop.org/software/systemd/man/…
– George Udosen
Dec 29 '18 at 6:11
add a comment |
When I look at journalctl
, it tells me the PID and the program name(or service name?) of a log entry.
Then I wondered, logs are created by other processes, how do systemd-journald
know the PID of these processes when processes may only write raw strings to the unix domain socket which systemd-journald
is listenning. Also, do sytemd-journald
always use the same technique to detect the PID of a piece of log data even when processes are producing log using functions like sd_journal_sendv()
?
Is there any documentation I should read about this?
I read JdeBP's answer and know systemd-journald
listen on an Unix Domian Socket, but even if can know the peer socket address who send the log message, how does it know the PID? What if that sending socket is opened by many non-parent-children processes?
systemd-journald ipc
When I look at journalctl
, it tells me the PID and the program name(or service name?) of a log entry.
Then I wondered, logs are created by other processes, how do systemd-journald
know the PID of these processes when processes may only write raw strings to the unix domain socket which systemd-journald
is listenning. Also, do sytemd-journald
always use the same technique to detect the PID of a piece of log data even when processes are producing log using functions like sd_journal_sendv()
?
Is there any documentation I should read about this?
I read JdeBP's answer and know systemd-journald
listen on an Unix Domian Socket, but even if can know the peer socket address who send the log message, how does it know the PID? What if that sending socket is opened by many non-parent-children processes?
systemd-journald ipc
systemd-journald ipc
edited Dec 29 '18 at 6:25
asked Dec 29 '18 at 5:51
炸鱼薯条德里克
428114
428114
See: freedesktop.org/software/systemd/man/…
– George Udosen
Dec 29 '18 at 6:11
add a comment |
See: freedesktop.org/software/systemd/man/…
– George Udosen
Dec 29 '18 at 6:11
See: freedesktop.org/software/systemd/man/…
– George Udosen
Dec 29 '18 at 6:11
See: freedesktop.org/software/systemd/man/…
– George Udosen
Dec 29 '18 at 6:11
add a comment |
2 Answers
2
active
oldest
votes
It receives the pid via the SCM_CREDENTIALS
ancillary data on the unix socket with recvmsg()
, see unix(7)
. The credentials don't have to be sent explicitly.
Example:
$ cc -Wall scm_cred.c -o scm_cred
$ ./scm_cred
scm_cred: received from 10114: pid=10114 uid=2000 gid=2000
Processes with CAP_SYS_ADMIN
data can send whatever pid they want via SCM_CREDENTIALS
; in the case of systemd-journald
, this means they can fake entries as if logged by another process:
# cc -Wall fake.c -o fake
# setcap CAP_SYS_ADMIN+ep fake
$ ./fake `pgrep -f /usr/sbin/sshd`
# journalctl --no-pager -n 1
...
Dec 29 11:04:57 debin sshd[419]: fake log message from 14202
# rm fake
# lsb_release -d
Description: Debian GNU/Linux 9.6 (stretch)
systemd-journald
handles datagrams and credentials sent via ancillary data is in the server_process_datagram()
function from journald-server.c
. Both the syslog(3)
standard function from libc
and sd_journal_sendv()
from libsystemd
will send their data via a SOCK_DGRAM
socket by default, and getsockopt(SO_PEERCRED)
does not work on datagram (connectionless) sockets. Neither systemd-journald
nor rsyslogd
accept SOCK_STREAM
connections on /dev/log
.
scm_cred.c
#define _GNU_SOURCE 1
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#include <err.h>
int main(void){
int fd[2]; pid_t pid;
if(socketpair(AF_LOCAL, SOCK_DGRAM, 0, fd)) err(1, "socketpair");
if((pid = fork()) == -1) err(1, "fork");
if(pid){ /* parent */
int on = 1;
union {
struct cmsghdr h;
char data[CMSG_SPACE(sizeof(struct ucred))];
} buf;
struct msghdr m = {0};
struct ucred *uc = (struct ucred*)CMSG_DATA(&buf.h);
m.msg_control = &buf;
m.msg_controllen = sizeof buf;
if(setsockopt(fd[0], SOL_SOCKET, SO_PASSCRED, &on, sizeof on))
err(1, "setsockopt");
if(recvmsg(fd[0], &m, 0) == -1) err(1, "recvmsg");
warnx("received from %d: pid=%d uid=%d gid=%d", pid,
uc->pid, uc->uid, uc->gid);
}else /* child */
write(fd[1], 0, 0);
return 0;
}
fake.c
#define _GNU_SOURCE 1
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <err.h>
int main(int ac, char **av){
union {
struct cmsghdr h;
char data[CMSG_SPACE(sizeof(struct ucred))];
} cm;
int fd; char buf[256];
struct ucred *uc = (struct ucred*)CMSG_DATA(&cm.h);
struct msghdr m = {0};
struct sockaddr_un ua = {AF_UNIX, "/dev/log"};
struct iovec iov = {buf};
if((fd = socket(AF_LOCAL, SOCK_DGRAM, 0)) == -1) err(1, "socket");
if(connect(fd, (struct sockaddr*)&ua, SUN_LEN(&ua))) err(1, "connect");
m.msg_control = &cm;
m.msg_controllen = cm.h.cmsg_len = CMSG_LEN(sizeof(struct ucred));
cm.h.cmsg_level = SOL_SOCKET;
cm.h.cmsg_type = SCM_CREDENTIALS;
uc->pid = ac > 1 ? atoi(av[1]) : getpid();
uc->uid = ac > 2 ? atoi(av[2]) : geteuid();
uc->gid = ac > 3 ? atoi(av[3]) : getegid();
iov.iov_len = snprintf(buf, sizeof buf, "<13>%s from %d",
ac > 4 ? av[4] : "fake log message", getpid());
if(iov.iov_len >= sizeof buf) errx(1, "message too long");
m.msg_iov = &iov;
m.msg_iovlen = 1;
if(sendmsg(fd, &m, 0) == -1) err(1, "sendmsg");
return 0;
}
I see. So get the sender's info doesn't need the sender process to send it initiatively. But what if the sender process has CAP_SYS_ADMIN andsendmsg()
a PID different from its own? Willsystemd-journald
get tricked by this behaviour?
– 炸鱼薯条德里克
Dec 29 '18 at 7:36
no, the kernel checks the credentials. that's mentioned in the unix(7) manpage under SCM_CREDENTIALS.
– mosvy
Dec 29 '18 at 7:38
Yeah, but it mentionedThe sender must specify its own process ID (unless it has the capability CAP_SYS_ADMIN)
. That's why I mention the CAP_SYS_ADMIN, am I misunderstanding anything?
– 炸鱼薯条德里克
Dec 29 '18 at 7:40
Yes, a process withCAP_SYS_ADMIN
can send a pid different from its own. (Haven't tested it, though)
– mosvy
Dec 29 '18 at 7:43
add a comment |
The kernel tells it.
The EUID, EGID, and PID of the original client process that connected the AF_LOCAL
stream socket at /run/systemd/journal/stdout
is available from the kernel via the SO_PEERCRED
socket option, which it uses. UCSPI-UNIX tools obtain this same information via the same system call.
Child service processes of course inherit their standard I/O file descriptors already opened (unless the parent service process changes this, of course), and so to systemd-journald
all log output has the credentials of the original parent process.
Log output generated via the AF_LOCAL
socket at /run/systemd/journal/socket
that speaks the idiosyncratic systemd-journald
protocol is coming over a datagram socket, rather than a stream one. This socket is flagged using the SO_PASSCRED
socket option so that the kernel records the same information in each datagram sent, which is pulled out of each datagram by systemd-journald
.
Further reading
getsockopt()
. Linux Programmers' Manual. 2017-09-15.
socket
. Linux Programmers' Manual. 2018-02-02.- Jonathan de Boyne Pollard (2017).
local-stream-socket-accept
. nosh Guide. Softwares. - Jonathan de Boyne Pollard (2015). "Environment variables". The gen on the UNIX Client-Server Program Interface. Frequently Given Answers.
no, it doesn't get it viaSO_PEERCRED
, but via ancillary data withrecvmsg
. I'vestrace
'dsystemd-journald
.
– mosvy
Dec 29 '18 at 8:10
… and you haven't read what you are commenting on, or my previous answer referred to in the question, or indeed all of what the question asks.
– JdeBP
Dec 29 '18 at 17:05
Because the rude dress-down may give the wrong impressions wrt the accuracy of this answer, I want to make it clear: this answer is wrong. I'll try to explain why. 1. Portable apps which are usingsyslog()
do not connect to the stream socket from/run/systemd/journal/stdout
; they simply send their data from an unconnected, datagram to/dev/log
. SinceSO_PEERCRED
is getting the creds of the process that connected to a socket, and does not work with connectionless sockets, it cannot be and is not used to get the pid of the process that calledsyslog()
.
– mosvy
Dec 30 '18 at 8:01
2. unless overrided by a privileged process, the creds thatsystemd
gets viarecvmsg
as described in my answer will be those of the process that calledsend()
by way ofsyslog()
, not of the parent process that created the socket or calledconnect()
on it. The second paragraph is particularly misleading, because evenSO_PEERCRED
on a connection-based socket will not return the creds of the process that created the socket file descriptor, but of the process thatconnect()
ed it.
– mosvy
Dec 30 '18 at 8:02
3. TheSO_PASSCRED
option should be set on the socket on which the creds are to be received, not on the socket on which they're sent, and it does not cause the kernel to stick the same info in each datagram sent in the way it's described in the 3rd paragraph.
– mosvy
Dec 30 '18 at 8:03
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f491416%2fhow-does-journald-know-the-pid-of-a-process-that-produces-log-data%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
It receives the pid via the SCM_CREDENTIALS
ancillary data on the unix socket with recvmsg()
, see unix(7)
. The credentials don't have to be sent explicitly.
Example:
$ cc -Wall scm_cred.c -o scm_cred
$ ./scm_cred
scm_cred: received from 10114: pid=10114 uid=2000 gid=2000
Processes with CAP_SYS_ADMIN
data can send whatever pid they want via SCM_CREDENTIALS
; in the case of systemd-journald
, this means they can fake entries as if logged by another process:
# cc -Wall fake.c -o fake
# setcap CAP_SYS_ADMIN+ep fake
$ ./fake `pgrep -f /usr/sbin/sshd`
# journalctl --no-pager -n 1
...
Dec 29 11:04:57 debin sshd[419]: fake log message from 14202
# rm fake
# lsb_release -d
Description: Debian GNU/Linux 9.6 (stretch)
systemd-journald
handles datagrams and credentials sent via ancillary data is in the server_process_datagram()
function from journald-server.c
. Both the syslog(3)
standard function from libc
and sd_journal_sendv()
from libsystemd
will send their data via a SOCK_DGRAM
socket by default, and getsockopt(SO_PEERCRED)
does not work on datagram (connectionless) sockets. Neither systemd-journald
nor rsyslogd
accept SOCK_STREAM
connections on /dev/log
.
scm_cred.c
#define _GNU_SOURCE 1
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#include <err.h>
int main(void){
int fd[2]; pid_t pid;
if(socketpair(AF_LOCAL, SOCK_DGRAM, 0, fd)) err(1, "socketpair");
if((pid = fork()) == -1) err(1, "fork");
if(pid){ /* parent */
int on = 1;
union {
struct cmsghdr h;
char data[CMSG_SPACE(sizeof(struct ucred))];
} buf;
struct msghdr m = {0};
struct ucred *uc = (struct ucred*)CMSG_DATA(&buf.h);
m.msg_control = &buf;
m.msg_controllen = sizeof buf;
if(setsockopt(fd[0], SOL_SOCKET, SO_PASSCRED, &on, sizeof on))
err(1, "setsockopt");
if(recvmsg(fd[0], &m, 0) == -1) err(1, "recvmsg");
warnx("received from %d: pid=%d uid=%d gid=%d", pid,
uc->pid, uc->uid, uc->gid);
}else /* child */
write(fd[1], 0, 0);
return 0;
}
fake.c
#define _GNU_SOURCE 1
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <err.h>
int main(int ac, char **av){
union {
struct cmsghdr h;
char data[CMSG_SPACE(sizeof(struct ucred))];
} cm;
int fd; char buf[256];
struct ucred *uc = (struct ucred*)CMSG_DATA(&cm.h);
struct msghdr m = {0};
struct sockaddr_un ua = {AF_UNIX, "/dev/log"};
struct iovec iov = {buf};
if((fd = socket(AF_LOCAL, SOCK_DGRAM, 0)) == -1) err(1, "socket");
if(connect(fd, (struct sockaddr*)&ua, SUN_LEN(&ua))) err(1, "connect");
m.msg_control = &cm;
m.msg_controllen = cm.h.cmsg_len = CMSG_LEN(sizeof(struct ucred));
cm.h.cmsg_level = SOL_SOCKET;
cm.h.cmsg_type = SCM_CREDENTIALS;
uc->pid = ac > 1 ? atoi(av[1]) : getpid();
uc->uid = ac > 2 ? atoi(av[2]) : geteuid();
uc->gid = ac > 3 ? atoi(av[3]) : getegid();
iov.iov_len = snprintf(buf, sizeof buf, "<13>%s from %d",
ac > 4 ? av[4] : "fake log message", getpid());
if(iov.iov_len >= sizeof buf) errx(1, "message too long");
m.msg_iov = &iov;
m.msg_iovlen = 1;
if(sendmsg(fd, &m, 0) == -1) err(1, "sendmsg");
return 0;
}
I see. So get the sender's info doesn't need the sender process to send it initiatively. But what if the sender process has CAP_SYS_ADMIN andsendmsg()
a PID different from its own? Willsystemd-journald
get tricked by this behaviour?
– 炸鱼薯条德里克
Dec 29 '18 at 7:36
no, the kernel checks the credentials. that's mentioned in the unix(7) manpage under SCM_CREDENTIALS.
– mosvy
Dec 29 '18 at 7:38
Yeah, but it mentionedThe sender must specify its own process ID (unless it has the capability CAP_SYS_ADMIN)
. That's why I mention the CAP_SYS_ADMIN, am I misunderstanding anything?
– 炸鱼薯条德里克
Dec 29 '18 at 7:40
Yes, a process withCAP_SYS_ADMIN
can send a pid different from its own. (Haven't tested it, though)
– mosvy
Dec 29 '18 at 7:43
add a comment |
It receives the pid via the SCM_CREDENTIALS
ancillary data on the unix socket with recvmsg()
, see unix(7)
. The credentials don't have to be sent explicitly.
Example:
$ cc -Wall scm_cred.c -o scm_cred
$ ./scm_cred
scm_cred: received from 10114: pid=10114 uid=2000 gid=2000
Processes with CAP_SYS_ADMIN
data can send whatever pid they want via SCM_CREDENTIALS
; in the case of systemd-journald
, this means they can fake entries as if logged by another process:
# cc -Wall fake.c -o fake
# setcap CAP_SYS_ADMIN+ep fake
$ ./fake `pgrep -f /usr/sbin/sshd`
# journalctl --no-pager -n 1
...
Dec 29 11:04:57 debin sshd[419]: fake log message from 14202
# rm fake
# lsb_release -d
Description: Debian GNU/Linux 9.6 (stretch)
systemd-journald
handles datagrams and credentials sent via ancillary data is in the server_process_datagram()
function from journald-server.c
. Both the syslog(3)
standard function from libc
and sd_journal_sendv()
from libsystemd
will send their data via a SOCK_DGRAM
socket by default, and getsockopt(SO_PEERCRED)
does not work on datagram (connectionless) sockets. Neither systemd-journald
nor rsyslogd
accept SOCK_STREAM
connections on /dev/log
.
scm_cred.c
#define _GNU_SOURCE 1
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#include <err.h>
int main(void){
int fd[2]; pid_t pid;
if(socketpair(AF_LOCAL, SOCK_DGRAM, 0, fd)) err(1, "socketpair");
if((pid = fork()) == -1) err(1, "fork");
if(pid){ /* parent */
int on = 1;
union {
struct cmsghdr h;
char data[CMSG_SPACE(sizeof(struct ucred))];
} buf;
struct msghdr m = {0};
struct ucred *uc = (struct ucred*)CMSG_DATA(&buf.h);
m.msg_control = &buf;
m.msg_controllen = sizeof buf;
if(setsockopt(fd[0], SOL_SOCKET, SO_PASSCRED, &on, sizeof on))
err(1, "setsockopt");
if(recvmsg(fd[0], &m, 0) == -1) err(1, "recvmsg");
warnx("received from %d: pid=%d uid=%d gid=%d", pid,
uc->pid, uc->uid, uc->gid);
}else /* child */
write(fd[1], 0, 0);
return 0;
}
fake.c
#define _GNU_SOURCE 1
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <err.h>
int main(int ac, char **av){
union {
struct cmsghdr h;
char data[CMSG_SPACE(sizeof(struct ucred))];
} cm;
int fd; char buf[256];
struct ucred *uc = (struct ucred*)CMSG_DATA(&cm.h);
struct msghdr m = {0};
struct sockaddr_un ua = {AF_UNIX, "/dev/log"};
struct iovec iov = {buf};
if((fd = socket(AF_LOCAL, SOCK_DGRAM, 0)) == -1) err(1, "socket");
if(connect(fd, (struct sockaddr*)&ua, SUN_LEN(&ua))) err(1, "connect");
m.msg_control = &cm;
m.msg_controllen = cm.h.cmsg_len = CMSG_LEN(sizeof(struct ucred));
cm.h.cmsg_level = SOL_SOCKET;
cm.h.cmsg_type = SCM_CREDENTIALS;
uc->pid = ac > 1 ? atoi(av[1]) : getpid();
uc->uid = ac > 2 ? atoi(av[2]) : geteuid();
uc->gid = ac > 3 ? atoi(av[3]) : getegid();
iov.iov_len = snprintf(buf, sizeof buf, "<13>%s from %d",
ac > 4 ? av[4] : "fake log message", getpid());
if(iov.iov_len >= sizeof buf) errx(1, "message too long");
m.msg_iov = &iov;
m.msg_iovlen = 1;
if(sendmsg(fd, &m, 0) == -1) err(1, "sendmsg");
return 0;
}
I see. So get the sender's info doesn't need the sender process to send it initiatively. But what if the sender process has CAP_SYS_ADMIN andsendmsg()
a PID different from its own? Willsystemd-journald
get tricked by this behaviour?
– 炸鱼薯条德里克
Dec 29 '18 at 7:36
no, the kernel checks the credentials. that's mentioned in the unix(7) manpage under SCM_CREDENTIALS.
– mosvy
Dec 29 '18 at 7:38
Yeah, but it mentionedThe sender must specify its own process ID (unless it has the capability CAP_SYS_ADMIN)
. That's why I mention the CAP_SYS_ADMIN, am I misunderstanding anything?
– 炸鱼薯条德里克
Dec 29 '18 at 7:40
Yes, a process withCAP_SYS_ADMIN
can send a pid different from its own. (Haven't tested it, though)
– mosvy
Dec 29 '18 at 7:43
add a comment |
It receives the pid via the SCM_CREDENTIALS
ancillary data on the unix socket with recvmsg()
, see unix(7)
. The credentials don't have to be sent explicitly.
Example:
$ cc -Wall scm_cred.c -o scm_cred
$ ./scm_cred
scm_cred: received from 10114: pid=10114 uid=2000 gid=2000
Processes with CAP_SYS_ADMIN
data can send whatever pid they want via SCM_CREDENTIALS
; in the case of systemd-journald
, this means they can fake entries as if logged by another process:
# cc -Wall fake.c -o fake
# setcap CAP_SYS_ADMIN+ep fake
$ ./fake `pgrep -f /usr/sbin/sshd`
# journalctl --no-pager -n 1
...
Dec 29 11:04:57 debin sshd[419]: fake log message from 14202
# rm fake
# lsb_release -d
Description: Debian GNU/Linux 9.6 (stretch)
systemd-journald
handles datagrams and credentials sent via ancillary data is in the server_process_datagram()
function from journald-server.c
. Both the syslog(3)
standard function from libc
and sd_journal_sendv()
from libsystemd
will send their data via a SOCK_DGRAM
socket by default, and getsockopt(SO_PEERCRED)
does not work on datagram (connectionless) sockets. Neither systemd-journald
nor rsyslogd
accept SOCK_STREAM
connections on /dev/log
.
scm_cred.c
#define _GNU_SOURCE 1
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#include <err.h>
int main(void){
int fd[2]; pid_t pid;
if(socketpair(AF_LOCAL, SOCK_DGRAM, 0, fd)) err(1, "socketpair");
if((pid = fork()) == -1) err(1, "fork");
if(pid){ /* parent */
int on = 1;
union {
struct cmsghdr h;
char data[CMSG_SPACE(sizeof(struct ucred))];
} buf;
struct msghdr m = {0};
struct ucred *uc = (struct ucred*)CMSG_DATA(&buf.h);
m.msg_control = &buf;
m.msg_controllen = sizeof buf;
if(setsockopt(fd[0], SOL_SOCKET, SO_PASSCRED, &on, sizeof on))
err(1, "setsockopt");
if(recvmsg(fd[0], &m, 0) == -1) err(1, "recvmsg");
warnx("received from %d: pid=%d uid=%d gid=%d", pid,
uc->pid, uc->uid, uc->gid);
}else /* child */
write(fd[1], 0, 0);
return 0;
}
fake.c
#define _GNU_SOURCE 1
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <err.h>
int main(int ac, char **av){
union {
struct cmsghdr h;
char data[CMSG_SPACE(sizeof(struct ucred))];
} cm;
int fd; char buf[256];
struct ucred *uc = (struct ucred*)CMSG_DATA(&cm.h);
struct msghdr m = {0};
struct sockaddr_un ua = {AF_UNIX, "/dev/log"};
struct iovec iov = {buf};
if((fd = socket(AF_LOCAL, SOCK_DGRAM, 0)) == -1) err(1, "socket");
if(connect(fd, (struct sockaddr*)&ua, SUN_LEN(&ua))) err(1, "connect");
m.msg_control = &cm;
m.msg_controllen = cm.h.cmsg_len = CMSG_LEN(sizeof(struct ucred));
cm.h.cmsg_level = SOL_SOCKET;
cm.h.cmsg_type = SCM_CREDENTIALS;
uc->pid = ac > 1 ? atoi(av[1]) : getpid();
uc->uid = ac > 2 ? atoi(av[2]) : geteuid();
uc->gid = ac > 3 ? atoi(av[3]) : getegid();
iov.iov_len = snprintf(buf, sizeof buf, "<13>%s from %d",
ac > 4 ? av[4] : "fake log message", getpid());
if(iov.iov_len >= sizeof buf) errx(1, "message too long");
m.msg_iov = &iov;
m.msg_iovlen = 1;
if(sendmsg(fd, &m, 0) == -1) err(1, "sendmsg");
return 0;
}
It receives the pid via the SCM_CREDENTIALS
ancillary data on the unix socket with recvmsg()
, see unix(7)
. The credentials don't have to be sent explicitly.
Example:
$ cc -Wall scm_cred.c -o scm_cred
$ ./scm_cred
scm_cred: received from 10114: pid=10114 uid=2000 gid=2000
Processes with CAP_SYS_ADMIN
data can send whatever pid they want via SCM_CREDENTIALS
; in the case of systemd-journald
, this means they can fake entries as if logged by another process:
# cc -Wall fake.c -o fake
# setcap CAP_SYS_ADMIN+ep fake
$ ./fake `pgrep -f /usr/sbin/sshd`
# journalctl --no-pager -n 1
...
Dec 29 11:04:57 debin sshd[419]: fake log message from 14202
# rm fake
# lsb_release -d
Description: Debian GNU/Linux 9.6 (stretch)
systemd-journald
handles datagrams and credentials sent via ancillary data is in the server_process_datagram()
function from journald-server.c
. Both the syslog(3)
standard function from libc
and sd_journal_sendv()
from libsystemd
will send their data via a SOCK_DGRAM
socket by default, and getsockopt(SO_PEERCRED)
does not work on datagram (connectionless) sockets. Neither systemd-journald
nor rsyslogd
accept SOCK_STREAM
connections on /dev/log
.
scm_cred.c
#define _GNU_SOURCE 1
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#include <err.h>
int main(void){
int fd[2]; pid_t pid;
if(socketpair(AF_LOCAL, SOCK_DGRAM, 0, fd)) err(1, "socketpair");
if((pid = fork()) == -1) err(1, "fork");
if(pid){ /* parent */
int on = 1;
union {
struct cmsghdr h;
char data[CMSG_SPACE(sizeof(struct ucred))];
} buf;
struct msghdr m = {0};
struct ucred *uc = (struct ucred*)CMSG_DATA(&buf.h);
m.msg_control = &buf;
m.msg_controllen = sizeof buf;
if(setsockopt(fd[0], SOL_SOCKET, SO_PASSCRED, &on, sizeof on))
err(1, "setsockopt");
if(recvmsg(fd[0], &m, 0) == -1) err(1, "recvmsg");
warnx("received from %d: pid=%d uid=%d gid=%d", pid,
uc->pid, uc->uid, uc->gid);
}else /* child */
write(fd[1], 0, 0);
return 0;
}
fake.c
#define _GNU_SOURCE 1
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <err.h>
int main(int ac, char **av){
union {
struct cmsghdr h;
char data[CMSG_SPACE(sizeof(struct ucred))];
} cm;
int fd; char buf[256];
struct ucred *uc = (struct ucred*)CMSG_DATA(&cm.h);
struct msghdr m = {0};
struct sockaddr_un ua = {AF_UNIX, "/dev/log"};
struct iovec iov = {buf};
if((fd = socket(AF_LOCAL, SOCK_DGRAM, 0)) == -1) err(1, "socket");
if(connect(fd, (struct sockaddr*)&ua, SUN_LEN(&ua))) err(1, "connect");
m.msg_control = &cm;
m.msg_controllen = cm.h.cmsg_len = CMSG_LEN(sizeof(struct ucred));
cm.h.cmsg_level = SOL_SOCKET;
cm.h.cmsg_type = SCM_CREDENTIALS;
uc->pid = ac > 1 ? atoi(av[1]) : getpid();
uc->uid = ac > 2 ? atoi(av[2]) : geteuid();
uc->gid = ac > 3 ? atoi(av[3]) : getegid();
iov.iov_len = snprintf(buf, sizeof buf, "<13>%s from %d",
ac > 4 ? av[4] : "fake log message", getpid());
if(iov.iov_len >= sizeof buf) errx(1, "message too long");
m.msg_iov = &iov;
m.msg_iovlen = 1;
if(sendmsg(fd, &m, 0) == -1) err(1, "sendmsg");
return 0;
}
edited Dec 29 '18 at 18:24
answered Dec 29 '18 at 7:04
mosvy
6,1761425
6,1761425
I see. So get the sender's info doesn't need the sender process to send it initiatively. But what if the sender process has CAP_SYS_ADMIN andsendmsg()
a PID different from its own? Willsystemd-journald
get tricked by this behaviour?
– 炸鱼薯条德里克
Dec 29 '18 at 7:36
no, the kernel checks the credentials. that's mentioned in the unix(7) manpage under SCM_CREDENTIALS.
– mosvy
Dec 29 '18 at 7:38
Yeah, but it mentionedThe sender must specify its own process ID (unless it has the capability CAP_SYS_ADMIN)
. That's why I mention the CAP_SYS_ADMIN, am I misunderstanding anything?
– 炸鱼薯条德里克
Dec 29 '18 at 7:40
Yes, a process withCAP_SYS_ADMIN
can send a pid different from its own. (Haven't tested it, though)
– mosvy
Dec 29 '18 at 7:43
add a comment |
I see. So get the sender's info doesn't need the sender process to send it initiatively. But what if the sender process has CAP_SYS_ADMIN andsendmsg()
a PID different from its own? Willsystemd-journald
get tricked by this behaviour?
– 炸鱼薯条德里克
Dec 29 '18 at 7:36
no, the kernel checks the credentials. that's mentioned in the unix(7) manpage under SCM_CREDENTIALS.
– mosvy
Dec 29 '18 at 7:38
Yeah, but it mentionedThe sender must specify its own process ID (unless it has the capability CAP_SYS_ADMIN)
. That's why I mention the CAP_SYS_ADMIN, am I misunderstanding anything?
– 炸鱼薯条德里克
Dec 29 '18 at 7:40
Yes, a process withCAP_SYS_ADMIN
can send a pid different from its own. (Haven't tested it, though)
– mosvy
Dec 29 '18 at 7:43
I see. So get the sender's info doesn't need the sender process to send it initiatively. But what if the sender process has CAP_SYS_ADMIN and
sendmsg()
a PID different from its own? Will systemd-journald
get tricked by this behaviour?– 炸鱼薯条德里克
Dec 29 '18 at 7:36
I see. So get the sender's info doesn't need the sender process to send it initiatively. But what if the sender process has CAP_SYS_ADMIN and
sendmsg()
a PID different from its own? Will systemd-journald
get tricked by this behaviour?– 炸鱼薯条德里克
Dec 29 '18 at 7:36
no, the kernel checks the credentials. that's mentioned in the unix(7) manpage under SCM_CREDENTIALS.
– mosvy
Dec 29 '18 at 7:38
no, the kernel checks the credentials. that's mentioned in the unix(7) manpage under SCM_CREDENTIALS.
– mosvy
Dec 29 '18 at 7:38
Yeah, but it mentioned
The sender must specify its own process ID (unless it has the capability CAP_SYS_ADMIN)
. That's why I mention the CAP_SYS_ADMIN, am I misunderstanding anything?– 炸鱼薯条德里克
Dec 29 '18 at 7:40
Yeah, but it mentioned
The sender must specify its own process ID (unless it has the capability CAP_SYS_ADMIN)
. That's why I mention the CAP_SYS_ADMIN, am I misunderstanding anything?– 炸鱼薯条德里克
Dec 29 '18 at 7:40
Yes, a process with
CAP_SYS_ADMIN
can send a pid different from its own. (Haven't tested it, though)– mosvy
Dec 29 '18 at 7:43
Yes, a process with
CAP_SYS_ADMIN
can send a pid different from its own. (Haven't tested it, though)– mosvy
Dec 29 '18 at 7:43
add a comment |
The kernel tells it.
The EUID, EGID, and PID of the original client process that connected the AF_LOCAL
stream socket at /run/systemd/journal/stdout
is available from the kernel via the SO_PEERCRED
socket option, which it uses. UCSPI-UNIX tools obtain this same information via the same system call.
Child service processes of course inherit their standard I/O file descriptors already opened (unless the parent service process changes this, of course), and so to systemd-journald
all log output has the credentials of the original parent process.
Log output generated via the AF_LOCAL
socket at /run/systemd/journal/socket
that speaks the idiosyncratic systemd-journald
protocol is coming over a datagram socket, rather than a stream one. This socket is flagged using the SO_PASSCRED
socket option so that the kernel records the same information in each datagram sent, which is pulled out of each datagram by systemd-journald
.
Further reading
getsockopt()
. Linux Programmers' Manual. 2017-09-15.
socket
. Linux Programmers' Manual. 2018-02-02.- Jonathan de Boyne Pollard (2017).
local-stream-socket-accept
. nosh Guide. Softwares. - Jonathan de Boyne Pollard (2015). "Environment variables". The gen on the UNIX Client-Server Program Interface. Frequently Given Answers.
no, it doesn't get it viaSO_PEERCRED
, but via ancillary data withrecvmsg
. I'vestrace
'dsystemd-journald
.
– mosvy
Dec 29 '18 at 8:10
… and you haven't read what you are commenting on, or my previous answer referred to in the question, or indeed all of what the question asks.
– JdeBP
Dec 29 '18 at 17:05
Because the rude dress-down may give the wrong impressions wrt the accuracy of this answer, I want to make it clear: this answer is wrong. I'll try to explain why. 1. Portable apps which are usingsyslog()
do not connect to the stream socket from/run/systemd/journal/stdout
; they simply send their data from an unconnected, datagram to/dev/log
. SinceSO_PEERCRED
is getting the creds of the process that connected to a socket, and does not work with connectionless sockets, it cannot be and is not used to get the pid of the process that calledsyslog()
.
– mosvy
Dec 30 '18 at 8:01
2. unless overrided by a privileged process, the creds thatsystemd
gets viarecvmsg
as described in my answer will be those of the process that calledsend()
by way ofsyslog()
, not of the parent process that created the socket or calledconnect()
on it. The second paragraph is particularly misleading, because evenSO_PEERCRED
on a connection-based socket will not return the creds of the process that created the socket file descriptor, but of the process thatconnect()
ed it.
– mosvy
Dec 30 '18 at 8:02
3. TheSO_PASSCRED
option should be set on the socket on which the creds are to be received, not on the socket on which they're sent, and it does not cause the kernel to stick the same info in each datagram sent in the way it's described in the 3rd paragraph.
– mosvy
Dec 30 '18 at 8:03
add a comment |
The kernel tells it.
The EUID, EGID, and PID of the original client process that connected the AF_LOCAL
stream socket at /run/systemd/journal/stdout
is available from the kernel via the SO_PEERCRED
socket option, which it uses. UCSPI-UNIX tools obtain this same information via the same system call.
Child service processes of course inherit their standard I/O file descriptors already opened (unless the parent service process changes this, of course), and so to systemd-journald
all log output has the credentials of the original parent process.
Log output generated via the AF_LOCAL
socket at /run/systemd/journal/socket
that speaks the idiosyncratic systemd-journald
protocol is coming over a datagram socket, rather than a stream one. This socket is flagged using the SO_PASSCRED
socket option so that the kernel records the same information in each datagram sent, which is pulled out of each datagram by systemd-journald
.
Further reading
getsockopt()
. Linux Programmers' Manual. 2017-09-15.
socket
. Linux Programmers' Manual. 2018-02-02.- Jonathan de Boyne Pollard (2017).
local-stream-socket-accept
. nosh Guide. Softwares. - Jonathan de Boyne Pollard (2015). "Environment variables". The gen on the UNIX Client-Server Program Interface. Frequently Given Answers.
no, it doesn't get it viaSO_PEERCRED
, but via ancillary data withrecvmsg
. I'vestrace
'dsystemd-journald
.
– mosvy
Dec 29 '18 at 8:10
… and you haven't read what you are commenting on, or my previous answer referred to in the question, or indeed all of what the question asks.
– JdeBP
Dec 29 '18 at 17:05
Because the rude dress-down may give the wrong impressions wrt the accuracy of this answer, I want to make it clear: this answer is wrong. I'll try to explain why. 1. Portable apps which are usingsyslog()
do not connect to the stream socket from/run/systemd/journal/stdout
; they simply send their data from an unconnected, datagram to/dev/log
. SinceSO_PEERCRED
is getting the creds of the process that connected to a socket, and does not work with connectionless sockets, it cannot be and is not used to get the pid of the process that calledsyslog()
.
– mosvy
Dec 30 '18 at 8:01
2. unless overrided by a privileged process, the creds thatsystemd
gets viarecvmsg
as described in my answer will be those of the process that calledsend()
by way ofsyslog()
, not of the parent process that created the socket or calledconnect()
on it. The second paragraph is particularly misleading, because evenSO_PEERCRED
on a connection-based socket will not return the creds of the process that created the socket file descriptor, but of the process thatconnect()
ed it.
– mosvy
Dec 30 '18 at 8:02
3. TheSO_PASSCRED
option should be set on the socket on which the creds are to be received, not on the socket on which they're sent, and it does not cause the kernel to stick the same info in each datagram sent in the way it's described in the 3rd paragraph.
– mosvy
Dec 30 '18 at 8:03
add a comment |
The kernel tells it.
The EUID, EGID, and PID of the original client process that connected the AF_LOCAL
stream socket at /run/systemd/journal/stdout
is available from the kernel via the SO_PEERCRED
socket option, which it uses. UCSPI-UNIX tools obtain this same information via the same system call.
Child service processes of course inherit their standard I/O file descriptors already opened (unless the parent service process changes this, of course), and so to systemd-journald
all log output has the credentials of the original parent process.
Log output generated via the AF_LOCAL
socket at /run/systemd/journal/socket
that speaks the idiosyncratic systemd-journald
protocol is coming over a datagram socket, rather than a stream one. This socket is flagged using the SO_PASSCRED
socket option so that the kernel records the same information in each datagram sent, which is pulled out of each datagram by systemd-journald
.
Further reading
getsockopt()
. Linux Programmers' Manual. 2017-09-15.
socket
. Linux Programmers' Manual. 2018-02-02.- Jonathan de Boyne Pollard (2017).
local-stream-socket-accept
. nosh Guide. Softwares. - Jonathan de Boyne Pollard (2015). "Environment variables". The gen on the UNIX Client-Server Program Interface. Frequently Given Answers.
The kernel tells it.
The EUID, EGID, and PID of the original client process that connected the AF_LOCAL
stream socket at /run/systemd/journal/stdout
is available from the kernel via the SO_PEERCRED
socket option, which it uses. UCSPI-UNIX tools obtain this same information via the same system call.
Child service processes of course inherit their standard I/O file descriptors already opened (unless the parent service process changes this, of course), and so to systemd-journald
all log output has the credentials of the original parent process.
Log output generated via the AF_LOCAL
socket at /run/systemd/journal/socket
that speaks the idiosyncratic systemd-journald
protocol is coming over a datagram socket, rather than a stream one. This socket is flagged using the SO_PASSCRED
socket option so that the kernel records the same information in each datagram sent, which is pulled out of each datagram by systemd-journald
.
Further reading
getsockopt()
. Linux Programmers' Manual. 2017-09-15.
socket
. Linux Programmers' Manual. 2018-02-02.- Jonathan de Boyne Pollard (2017).
local-stream-socket-accept
. nosh Guide. Softwares. - Jonathan de Boyne Pollard (2015). "Environment variables". The gen on the UNIX Client-Server Program Interface. Frequently Given Answers.
answered Dec 29 '18 at 7:56
JdeBP
33.5k469157
33.5k469157
no, it doesn't get it viaSO_PEERCRED
, but via ancillary data withrecvmsg
. I'vestrace
'dsystemd-journald
.
– mosvy
Dec 29 '18 at 8:10
… and you haven't read what you are commenting on, or my previous answer referred to in the question, or indeed all of what the question asks.
– JdeBP
Dec 29 '18 at 17:05
Because the rude dress-down may give the wrong impressions wrt the accuracy of this answer, I want to make it clear: this answer is wrong. I'll try to explain why. 1. Portable apps which are usingsyslog()
do not connect to the stream socket from/run/systemd/journal/stdout
; they simply send their data from an unconnected, datagram to/dev/log
. SinceSO_PEERCRED
is getting the creds of the process that connected to a socket, and does not work with connectionless sockets, it cannot be and is not used to get the pid of the process that calledsyslog()
.
– mosvy
Dec 30 '18 at 8:01
2. unless overrided by a privileged process, the creds thatsystemd
gets viarecvmsg
as described in my answer will be those of the process that calledsend()
by way ofsyslog()
, not of the parent process that created the socket or calledconnect()
on it. The second paragraph is particularly misleading, because evenSO_PEERCRED
on a connection-based socket will not return the creds of the process that created the socket file descriptor, but of the process thatconnect()
ed it.
– mosvy
Dec 30 '18 at 8:02
3. TheSO_PASSCRED
option should be set on the socket on which the creds are to be received, not on the socket on which they're sent, and it does not cause the kernel to stick the same info in each datagram sent in the way it's described in the 3rd paragraph.
– mosvy
Dec 30 '18 at 8:03
add a comment |
no, it doesn't get it viaSO_PEERCRED
, but via ancillary data withrecvmsg
. I'vestrace
'dsystemd-journald
.
– mosvy
Dec 29 '18 at 8:10
… and you haven't read what you are commenting on, or my previous answer referred to in the question, or indeed all of what the question asks.
– JdeBP
Dec 29 '18 at 17:05
Because the rude dress-down may give the wrong impressions wrt the accuracy of this answer, I want to make it clear: this answer is wrong. I'll try to explain why. 1. Portable apps which are usingsyslog()
do not connect to the stream socket from/run/systemd/journal/stdout
; they simply send their data from an unconnected, datagram to/dev/log
. SinceSO_PEERCRED
is getting the creds of the process that connected to a socket, and does not work with connectionless sockets, it cannot be and is not used to get the pid of the process that calledsyslog()
.
– mosvy
Dec 30 '18 at 8:01
2. unless overrided by a privileged process, the creds thatsystemd
gets viarecvmsg
as described in my answer will be those of the process that calledsend()
by way ofsyslog()
, not of the parent process that created the socket or calledconnect()
on it. The second paragraph is particularly misleading, because evenSO_PEERCRED
on a connection-based socket will not return the creds of the process that created the socket file descriptor, but of the process thatconnect()
ed it.
– mosvy
Dec 30 '18 at 8:02
3. TheSO_PASSCRED
option should be set on the socket on which the creds are to be received, not on the socket on which they're sent, and it does not cause the kernel to stick the same info in each datagram sent in the way it's described in the 3rd paragraph.
– mosvy
Dec 30 '18 at 8:03
no, it doesn't get it via
SO_PEERCRED
, but via ancillary data with recvmsg
. I've strace
'd systemd-journald
.– mosvy
Dec 29 '18 at 8:10
no, it doesn't get it via
SO_PEERCRED
, but via ancillary data with recvmsg
. I've strace
'd systemd-journald
.– mosvy
Dec 29 '18 at 8:10
… and you haven't read what you are commenting on, or my previous answer referred to in the question, or indeed all of what the question asks.
– JdeBP
Dec 29 '18 at 17:05
… and you haven't read what you are commenting on, or my previous answer referred to in the question, or indeed all of what the question asks.
– JdeBP
Dec 29 '18 at 17:05
Because the rude dress-down may give the wrong impressions wrt the accuracy of this answer, I want to make it clear: this answer is wrong. I'll try to explain why. 1. Portable apps which are using
syslog()
do not connect to the stream socket from /run/systemd/journal/stdout
; they simply send their data from an unconnected, datagram to /dev/log
. Since SO_PEERCRED
is getting the creds of the process that connected to a socket, and does not work with connectionless sockets, it cannot be and is not used to get the pid of the process that called syslog()
.– mosvy
Dec 30 '18 at 8:01
Because the rude dress-down may give the wrong impressions wrt the accuracy of this answer, I want to make it clear: this answer is wrong. I'll try to explain why. 1. Portable apps which are using
syslog()
do not connect to the stream socket from /run/systemd/journal/stdout
; they simply send their data from an unconnected, datagram to /dev/log
. Since SO_PEERCRED
is getting the creds of the process that connected to a socket, and does not work with connectionless sockets, it cannot be and is not used to get the pid of the process that called syslog()
.– mosvy
Dec 30 '18 at 8:01
2. unless overrided by a privileged process, the creds that
systemd
gets via recvmsg
as described in my answer will be those of the process that called send()
by way of syslog()
, not of the parent process that created the socket or called connect()
on it. The second paragraph is particularly misleading, because even SO_PEERCRED
on a connection-based socket will not return the creds of the process that created the socket file descriptor, but of the process that connect()
ed it.– mosvy
Dec 30 '18 at 8:02
2. unless overrided by a privileged process, the creds that
systemd
gets via recvmsg
as described in my answer will be those of the process that called send()
by way of syslog()
, not of the parent process that created the socket or called connect()
on it. The second paragraph is particularly misleading, because even SO_PEERCRED
on a connection-based socket will not return the creds of the process that created the socket file descriptor, but of the process that connect()
ed it.– mosvy
Dec 30 '18 at 8:02
3. The
SO_PASSCRED
option should be set on the socket on which the creds are to be received, not on the socket on which they're sent, and it does not cause the kernel to stick the same info in each datagram sent in the way it's described in the 3rd paragraph.– mosvy
Dec 30 '18 at 8:03
3. The
SO_PASSCRED
option should be set on the socket on which the creds are to be received, not on the socket on which they're sent, and it does not cause the kernel to stick the same info in each datagram sent in the way it's described in the 3rd paragraph.– mosvy
Dec 30 '18 at 8:03
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f491416%2fhow-does-journald-know-the-pid-of-a-process-that-produces-log-data%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
See: freedesktop.org/software/systemd/man/…
– George Udosen
Dec 29 '18 at 6:11