Run a program and log to both screen and file in real time
I've written a very simple python program (in reality my program is a lot more complex):
from time import sleep
for i in range(10):
print(i)
sleep(1)
to print the digits 0-9, 1 each second over 10 seconds. I've saved it as TestShell.py
I can run it in powershell using
python .TestShell.py
which outputs correctly, 1 digit each second. However I'd also like to store the output in a log file. I expected to use the Tee-Object for this
python .TestShell.py | Tee-Object -FilePath .fileLog.txt
which takes the log file as one parameter, and implicitly writes to console without a 2nd parameter. However now, no output is printed until the entire program is run. That is, I see nothing for 10 seconds, then the numbers 0-9 all at once.
Is there any way to run a (python) file, outputting to the terminal and a log file in real time?
windows-10 powershell python
asked Nov 17 '18 at 14:18
Greedo
235415
add a comment |
I've written a very simple python program (in reality my program is a lot more complex):
from time import sleep
for i in range(10):
print(i)
sleep(1)
to print the digits 0-9, 1 each second over 10 seconds. I've saved it as TestShell.py
I can run it in powershell using
python .TestShell.py
which outputs correctly, 1 digit each second. However I'd also like to store the output in a log file. I expected to use the Tee-Object for this
python .TestShell.py | Tee-Object -FilePath .fileLog.txt
which takes the log file as one parameter, and implicitly writes to console without a 2nd parameter. However now, no output is printed until the entire program is run. That is, I see nothing for 10 seconds, then the numbers 0-9 all at once.
Is there any way to run a (python) file, outputting to the terminal and a log file in real time?
windows-10 powershell python
asked Nov 17 '18 at 14:18
Greedo
235415
Honestly, I would add the logging in the script itself.
– LPChip
Nov 17 '18 at 14:37
1
@LPChip I thought about that, but in the real situation, most of the program output comes from a library I reference - I don't want to have to look inside it to change the logging. Even if I overload the default logger with a custom one, I have no way of guaranteeing the library isn't doing that too. So I really need a generic approach that monitors stdout and copies it to a file. I thought that was exactly whatteewould do - maybe some different parameters?
– Greedo
Nov 17 '18 at 14:43
add a comment |
I've written a very simple python program (in reality my program is a lot more complex):
from time import sleep
for i in range(10):
print(i)
sleep(1)
to print the digits 0-9, 1 each second over 10 seconds. I've saved it as TestShell.py
I can run it in powershell using
python .TestShell.py
which outputs correctly, 1 digit each second. However I'd also like to store the output in a log file. I expected to use the Tee-Object for this
python .TestShell.py | Tee-Object -FilePath .fileLog.txt
which takes the log file as one parameter, and implicitly writes to console without a 2nd parameter. However now, no output is printed until the entire program is run. That is, I see nothing for 10 seconds, then the numbers 0-9 all at once.
Is there any way to run a (python) file, outputting to the terminal and a log file in real time?
windows-10 powershell python
asked Nov 17 '18 at 14:18
Greedo
235415
I've written a very simple python program (in reality my program is a lot more complex):
from time import sleep
for i in range(10):
print(i)
sleep(1)
to print the digits 0-9, 1 each second over 10 seconds. I've saved it as TestShell.py
I can run it in powershell using
python .TestShell.py
which outputs correctly, 1 digit each second. However I'd also like to store the output in a log file. I expected to use the Tee-Object for this
python .TestShell.py | Tee-Object -FilePath .fileLog.txt
which takes the log file as one parameter, and implicitly writes to console without a 2nd parameter. However now, no output is printed until the entire program is run. That is, I see nothing for 10 seconds, then the numbers 0-9 all at once.
Is there any way to run a (python) file, outputting to the terminal and a log file in real time?
windows-10 powershell python
windows-10 powershell python
asked Nov 17 '18 at 14:18
Greedo
235415
asked Nov 17 '18 at 14:18
Greedo
235415
edited Nov 17 '18 at 14:44
asked Nov 17 '18 at 14:18
Greedo
235415
asked Nov 17 '18 at 14:18
Greedo
235415
asked Nov 17 '18 at 14:18
Greedo
235415
235415
Honestly, I would add the logging in the script itself.
– LPChip
Nov 17 '18 at 14:37
1
@LPChip I thought about that, but in the real situation, most of the program output comes from a library I reference - I don't want to have to look inside it to change the logging. Even if I overload the default logger with a custom one, I have no way of guaranteeing the library isn't doing that too. So I really need a generic approach that monitors stdout and copies it to a file. I thought that was exactly whatteewould do - maybe some different parameters?
– Greedo
Nov 17 '18 at 14:43
add a comment |
Honestly, I would add the logging in the script itself.
– LPChip
Nov 17 '18 at 14:37
1
@LPChip I thought about that, but in the real situation, most of the program output comes from a library I reference - I don't want to have to look inside it to change the logging. Even if I overload the default logger with a custom one, I have no way of guaranteeing the library isn't doing that too. So I really need a generic approach that monitors stdout and copies it to a file. I thought that was exactly whatteewould do - maybe some different parameters?
– Greedo
Nov 17 '18 at 14:43
Honestly, I would add the logging in the script itself.
– LPChip
Nov 17 '18 at 14:37
Honestly, I would add the logging in the script itself.
– LPChip
Nov 17 '18 at 14:37
1
1
@LPChip I thought about that, but in the real situation, most of the program output comes from a library I reference - I don't want to have to look inside it to change the logging. Even if I overload the default logger with a custom one, I have no way of guaranteeing the library isn't doing that too. So I really need a generic approach that monitors stdout and copies it to a file. I thought that was exactly what
tee would do - maybe some different parameters?– Greedo
Nov 17 '18 at 14:43
@LPChip I thought about that, but in the real situation, most of the program output comes from a library I reference - I don't want to have to look inside it to change the logging. Even if I overload the default logger with a custom one, I have no way of guaranteeing the library isn't doing that too. So I really need a generic approach that monitors stdout and copies it to a file. I thought that was exactly what
tee would do - maybe some different parameters?– Greedo
Nov 17 '18 at 14:43
add a comment |
2 Answers
2
active
oldest
votes
The problem is that the PowerShell Tee-Object doesn't flush the output stream,
but only waits for the source to do that, so you get the output at the end of the
operation. This is by design.
Use this syntax instead to write the output line-by-line :
python .TestShell.py | ForEach-Object { Write-Host $_; $_} | Set-Content .fileLog.txt
answered Nov 17 '18 at 14:50
harrymc
254k13265565
Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing"is a typo
– Greedo
Nov 17 '18 at 15:01
Is this an answer that is in the context to the original poster’s self answer?
– JakeGould
Nov 17 '18 at 15:01
Should be working. Strange.
– harrymc
Nov 17 '18 at 15:04
add a comment |
Well one approach can be found here
This is to use the -u switch to change the buffering mode of python itself, giving
python -u .TestShell.py | Tee-Object -FilePath .fileLog.txt
But that isn't a solution for all languages
answered Nov 17 '18 at 14:35
Greedo
235415
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "3"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1376241%2frun-a-program-and-log-to-both-screen-and-file-in-real-time%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
The problem is that the PowerShell Tee-Object doesn't flush the output stream,
but only waits for the source to do that, so you get the output at the end of the
operation. This is by design.
Use this syntax instead to write the output line-by-line :
python .TestShell.py | ForEach-Object { Write-Host $_; $_} | Set-Content .fileLog.txt
answered Nov 17 '18 at 14:50
harrymc
254k13265565
Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing"is a typo
– Greedo
Nov 17 '18 at 15:01
Is this an answer that is in the context to the original poster’s self answer?
– JakeGould
Nov 17 '18 at 15:01
Should be working. Strange.
– harrymc
Nov 17 '18 at 15:04
add a comment |
The problem is that the PowerShell Tee-Object doesn't flush the output stream,
but only waits for the source to do that, so you get the output at the end of the
operation. This is by design.
Use this syntax instead to write the output line-by-line :
python .TestShell.py | ForEach-Object { Write-Host $_; $_} | Set-Content .fileLog.txt
answered Nov 17 '18 at 14:50
harrymc
254k13265565
Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing"is a typo
– Greedo
Nov 17 '18 at 15:01
Is this an answer that is in the context to the original poster’s self answer?
– JakeGould
Nov 17 '18 at 15:01
Should be working. Strange.
– harrymc
Nov 17 '18 at 15:04
add a comment |
The problem is that the PowerShell Tee-Object doesn't flush the output stream,
but only waits for the source to do that, so you get the output at the end of the
operation. This is by design.
Use this syntax instead to write the output line-by-line :
python .TestShell.py | ForEach-Object { Write-Host $_; $_} | Set-Content .fileLog.txt
answered Nov 17 '18 at 14:50
harrymc
254k13265565
The problem is that the PowerShell Tee-Object doesn't flush the output stream,
but only waits for the source to do that, so you get the output at the end of the
operation. This is by design.
Use this syntax instead to write the output line-by-line :
python .TestShell.py | ForEach-Object { Write-Host $_; $_} | Set-Content .fileLog.txt
answered Nov 17 '18 at 14:50
harrymc
254k13265565
edited Nov 17 '18 at 15:03
answered Nov 17 '18 at 14:50
harrymc
254k13265565
answered Nov 17 '18 at 14:50
harrymc
254k13265565
answered Nov 17 '18 at 14:50
harrymc
254k13265565
254k13265565
Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing"is a typo
– Greedo
Nov 17 '18 at 15:01
Is this an answer that is in the context to the original poster’s self answer?
– JakeGould
Nov 17 '18 at 15:01
Should be working. Strange.
– harrymc
Nov 17 '18 at 15:04
add a comment |
Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing"is a typo
– Greedo
Nov 17 '18 at 15:01
Is this an answer that is in the context to the original poster’s self answer?
– JakeGould
Nov 17 '18 at 15:01
Should be working. Strange.
– harrymc
Nov 17 '18 at 15:04
Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing
" is a typo– Greedo
Nov 17 '18 at 15:01
Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing
" is a typo– Greedo
Nov 17 '18 at 15:01
Is this an answer that is in the context to the original poster’s self answer?
– JakeGould
Nov 17 '18 at 15:01
Is this an answer that is in the context to the original poster’s self answer?
– JakeGould
Nov 17 '18 at 15:01
Should be working. Strange.
– harrymc
Nov 17 '18 at 15:04
Should be working. Strange.
– harrymc
Nov 17 '18 at 15:04
add a comment |
Well one approach can be found here
This is to use the -u switch to change the buffering mode of python itself, giving
python -u .TestShell.py | Tee-Object -FilePath .fileLog.txt
But that isn't a solution for all languages
answered Nov 17 '18 at 14:35
Greedo
235415
add a comment |
Well one approach can be found here
This is to use the -u switch to change the buffering mode of python itself, giving
python -u .TestShell.py | Tee-Object -FilePath .fileLog.txt
But that isn't a solution for all languages
answered Nov 17 '18 at 14:35
Greedo
235415
add a comment |
Well one approach can be found here
This is to use the -u switch to change the buffering mode of python itself, giving
python -u .TestShell.py | Tee-Object -FilePath .fileLog.txt
But that isn't a solution for all languages
answered Nov 17 '18 at 14:35
Greedo
235415
Well one approach can be found here
This is to use the -u switch to change the buffering mode of python itself, giving
python -u .TestShell.py | Tee-Object -FilePath .fileLog.txt
But that isn't a solution for all languages
answered Nov 17 '18 at 14:35
Greedo
235415
edited Nov 17 '18 at 15:29
answered Nov 17 '18 at 14:35
Greedo
235415
answered Nov 17 '18 at 14:35
Greedo
235415
answered Nov 17 '18 at 14:35
Greedo
235415
235415
add a comment |
add a comment |
Thanks for contributing an answer to Super User!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1376241%2frun-a-program-and-log-to-both-screen-and-file-in-real-time%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Honestly, I would add the logging in the script itself.
– LPChip
Nov 17 '18 at 14:37
1
@LPChip I thought about that, but in the real situation, most of the program output comes from a library I reference - I don't want to have to look inside it to change the logging. Even if I overload the default logger with a custom one, I have no way of guaranteeing the library isn't doing that too. So I really need a generic approach that monitors stdout and copies it to a file. I thought that was exactly what
teewould do - maybe some different parameters?– Greedo
Nov 17 '18 at 14:43