python 3 Popen (semi-)interactive back-and-forth communication with a process, Popen.communicate() vs...
All of the examples I see for interacting with a process using Python 3's subprocess.Popen use Popen.communicate("input_text") exactly once before calling Popen.communicate() to grab the standard output and ending the program. I have a few programs that I want to script that require human intervention via stdin, so I want to automate them since the prompts are predictable.
For example, an in-house licensing application requires us to pass the application information via prompts (not from the command line) relating to the customer's unique ID (4 digit integer), the number of users, etc. And then it has to be done 30 times (random number), each one for a different product, identified by another integer.
Scripting that is easy if only I can learn how to do a sustained back-and-forth using Popen. Should I be using Popen.communicate() or should I be using Popen.stdout and Popen.stdin() and what's the difference between both?
python python-3.x subprocess popen
edited Dec 18 '18 at 20:51
ShadowRanger
62.9k66099
asked Nov 21 '18 at 18:12
UtahJarheadUtahJarhead
1,129717
add a comment |
All of the examples I see for interacting with a process using Python 3's subprocess.Popen use Popen.communicate("input_text") exactly once before calling Popen.communicate() to grab the standard output and ending the program. I have a few programs that I want to script that require human intervention via stdin, so I want to automate them since the prompts are predictable.
For example, an in-house licensing application requires us to pass the application information via prompts (not from the command line) relating to the customer's unique ID (4 digit integer), the number of users, etc. And then it has to be done 30 times (random number), each one for a different product, identified by another integer.
Scripting that is easy if only I can learn how to do a sustained back-and-forth using Popen. Should I be using Popen.communicate() or should I be using Popen.stdout and Popen.stdin() and what's the difference between both?
python python-3.x subprocess popen
edited Dec 18 '18 at 20:51
ShadowRanger
62.9k66099
asked Nov 21 '18 at 18:12
UtahJarheadUtahJarhead
1,129717
Do you know at launch time the full set of inputs the program will need, and can you wait to process results until the end? If so, stick withcommunicate, and feed it everything at once; it's simpler, and you don't risk deadlocks. If you can't predict the prompts, or must process stuff live, you're stuck with uglier code using either threads or theselect/selectorsmodule to avoid deadlocks.
– ShadowRanger
Dec 18 '18 at 20:50
add a comment |
All of the examples I see for interacting with a process using Python 3's subprocess.Popen use Popen.communicate("input_text") exactly once before calling Popen.communicate() to grab the standard output and ending the program. I have a few programs that I want to script that require human intervention via stdin, so I want to automate them since the prompts are predictable.
For example, an in-house licensing application requires us to pass the application information via prompts (not from the command line) relating to the customer's unique ID (4 digit integer), the number of users, etc. And then it has to be done 30 times (random number), each one for a different product, identified by another integer.
Scripting that is easy if only I can learn how to do a sustained back-and-forth using Popen. Should I be using Popen.communicate() or should I be using Popen.stdout and Popen.stdin() and what's the difference between both?
python python-3.x subprocess popen
edited Dec 18 '18 at 20:51
ShadowRanger
62.9k66099
asked Nov 21 '18 at 18:12
UtahJarheadUtahJarhead
1,129717
All of the examples I see for interacting with a process using Python 3's subprocess.Popen use Popen.communicate("input_text") exactly once before calling Popen.communicate() to grab the standard output and ending the program. I have a few programs that I want to script that require human intervention via stdin, so I want to automate them since the prompts are predictable.
For example, an in-house licensing application requires us to pass the application information via prompts (not from the command line) relating to the customer's unique ID (4 digit integer), the number of users, etc. And then it has to be done 30 times (random number), each one for a different product, identified by another integer.
Scripting that is easy if only I can learn how to do a sustained back-and-forth using Popen. Should I be using Popen.communicate() or should I be using Popen.stdout and Popen.stdin() and what's the difference between both?
python python-3.x subprocess popen
python python-3.x subprocess popen
edited Dec 18 '18 at 20:51
ShadowRanger
62.9k66099
asked Nov 21 '18 at 18:12
UtahJarheadUtahJarhead
1,129717
edited Dec 18 '18 at 20:51
ShadowRanger
62.9k66099
asked Nov 21 '18 at 18:12
UtahJarheadUtahJarhead
1,129717
edited Dec 18 '18 at 20:51
ShadowRanger
62.9k66099
edited Dec 18 '18 at 20:51
ShadowRanger
62.9k66099
edited Dec 18 '18 at 20:51
ShadowRanger
62.9k66099
62.9k66099
asked Nov 21 '18 at 18:12
UtahJarheadUtahJarhead
1,129717
asked Nov 21 '18 at 18:12
UtahJarheadUtahJarhead
1,129717
asked Nov 21 '18 at 18:12
UtahJarheadUtahJarhead
1,129717
1,129717
Do you know at launch time the full set of inputs the program will need, and can you wait to process results until the end? If so, stick withcommunicate, and feed it everything at once; it's simpler, and you don't risk deadlocks. If you can't predict the prompts, or must process stuff live, you're stuck with uglier code using either threads or theselect/selectorsmodule to avoid deadlocks.
– ShadowRanger
Dec 18 '18 at 20:50
add a comment |
Do you know at launch time the full set of inputs the program will need, and can you wait to process results until the end? If so, stick withcommunicate, and feed it everything at once; it's simpler, and you don't risk deadlocks. If you can't predict the prompts, or must process stuff live, you're stuck with uglier code using either threads or theselect/selectorsmodule to avoid deadlocks.
– ShadowRanger
Dec 18 '18 at 20:50
Do you know at launch time the full set of inputs the program will need, and can you wait to process results until the end? If so, stick with
communicate, and feed it everything at once; it's simpler, and you don't risk deadlocks. If you can't predict the prompts, or must process stuff live, you're stuck with uglier code using either threads or the select/selectors module to avoid deadlocks.– ShadowRanger
Dec 18 '18 at 20:50
Do you know at launch time the full set of inputs the program will need, and can you wait to process results until the end? If so, stick with
communicate, and feed it everything at once; it's simpler, and you don't risk deadlocks. If you can't predict the prompts, or must process stuff live, you're stuck with uglier code using either threads or the select/selectors module to avoid deadlocks.– ShadowRanger
Dec 18 '18 at 20:50
add a comment |
1 Answer
1
active
oldest
votes
Popen.communicate will block until the subprocess has completed or failed and only then returns information from stdout and stderr. So this is not what you need.
stdin, stdout & stderr are essentially special files belonging to a process that you can read from or write to, same as any other file, but they can provide an interface between processes if you pipe information into them.
I have recently had to implement something similar to what you described, the only way I was able to retrieve information through stdout of the "client" process was by using the pty module. I will link you two answers that helped me, however please note that these solutions are Posix only and using shell=True is a security risk. https://stackoverflow.com/a/5413588/533362 & https://stackoverflow.com/a/13605804/3565382.
answered Dec 18 '18 at 20:22
tmccaffreytmccaffrey
112
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53418210%2fpython-3-popen-semi-interactive-back-and-forth-communication-with-a-process-p%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Popen.communicate will block until the subprocess has completed or failed and only then returns information from stdout and stderr. So this is not what you need.
stdin, stdout & stderr are essentially special files belonging to a process that you can read from or write to, same as any other file, but they can provide an interface between processes if you pipe information into them.
I have recently had to implement something similar to what you described, the only way I was able to retrieve information through stdout of the "client" process was by using the pty module. I will link you two answers that helped me, however please note that these solutions are Posix only and using shell=True is a security risk. https://stackoverflow.com/a/5413588/533362 & https://stackoverflow.com/a/13605804/3565382.
answered Dec 18 '18 at 20:22
tmccaffreytmccaffrey
112
add a comment |
Popen.communicate will block until the subprocess has completed or failed and only then returns information from stdout and stderr. So this is not what you need.
stdin, stdout & stderr are essentially special files belonging to a process that you can read from or write to, same as any other file, but they can provide an interface between processes if you pipe information into them.
I have recently had to implement something similar to what you described, the only way I was able to retrieve information through stdout of the "client" process was by using the pty module. I will link you two answers that helped me, however please note that these solutions are Posix only and using shell=True is a security risk. https://stackoverflow.com/a/5413588/533362 & https://stackoverflow.com/a/13605804/3565382.
answered Dec 18 '18 at 20:22
tmccaffreytmccaffrey
112
add a comment |
Popen.communicate will block until the subprocess has completed or failed and only then returns information from stdout and stderr. So this is not what you need.
stdin, stdout & stderr are essentially special files belonging to a process that you can read from or write to, same as any other file, but they can provide an interface between processes if you pipe information into them.
I have recently had to implement something similar to what you described, the only way I was able to retrieve information through stdout of the "client" process was by using the pty module. I will link you two answers that helped me, however please note that these solutions are Posix only and using shell=True is a security risk. https://stackoverflow.com/a/5413588/533362 & https://stackoverflow.com/a/13605804/3565382.
answered Dec 18 '18 at 20:22
tmccaffreytmccaffrey
112
Popen.communicate will block until the subprocess has completed or failed and only then returns information from stdout and stderr. So this is not what you need.
stdin, stdout & stderr are essentially special files belonging to a process that you can read from or write to, same as any other file, but they can provide an interface between processes if you pipe information into them.
I have recently had to implement something similar to what you described, the only way I was able to retrieve information through stdout of the "client" process was by using the pty module. I will link you two answers that helped me, however please note that these solutions are Posix only and using shell=True is a security risk. https://stackoverflow.com/a/5413588/533362 & https://stackoverflow.com/a/13605804/3565382.
answered Dec 18 '18 at 20:22
tmccaffreytmccaffrey
112
answered Dec 18 '18 at 20:22
tmccaffreytmccaffrey
112
answered Dec 18 '18 at 20:22
tmccaffreytmccaffrey
112
answered Dec 18 '18 at 20:22
tmccaffreytmccaffrey
112
112
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53418210%2fpython-3-popen-semi-interactive-back-and-forth-communication-with-a-process-p%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Do you know at launch time the full set of inputs the program will need, and can you wait to process results until the end? If so, stick with
communicate, and feed it everything at once; it's simpler, and you don't risk deadlocks. If you can't predict the prompts, or must process stuff live, you're stuck with uglier code using either threads or theselect/selectorsmodule to avoid deadlocks.– ShadowRanger
Dec 18 '18 at 20:50