Why is .bash_history periodically wiped?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}
This is the second time it's happened.
I just tried grep-ing some historical commands and came up empty. a look at my bash history shows that it's practically empty. Like it's been wiped clean.
I use iTerm2 and on MacOS 10.13.6.
I see there's a .bash_sessions directory with some sessions saved but I guess this is probably an iTerm2 thing to preserve sessions for some reason or another.
terminal bash iterm
add a comment |
This is the second time it's happened.
I just tried grep-ing some historical commands and came up empty. a look at my bash history shows that it's practically empty. Like it's been wiped clean.
I use iTerm2 and on MacOS 10.13.6.
I see there's a .bash_sessions directory with some sessions saved but I guess this is probably an iTerm2 thing to preserve sessions for some reason or another.
terminal bash iterm
unix.stackexchange.com/questions/163371/…
– JBis
Mar 25 at 11:32
add a comment |
This is the second time it's happened.
I just tried grep-ing some historical commands and came up empty. a look at my bash history shows that it's practically empty. Like it's been wiped clean.
I use iTerm2 and on MacOS 10.13.6.
I see there's a .bash_sessions directory with some sessions saved but I guess this is probably an iTerm2 thing to preserve sessions for some reason or another.
terminal bash iterm
This is the second time it's happened.
I just tried grep-ing some historical commands and came up empty. a look at my bash history shows that it's practically empty. Like it's been wiped clean.
I use iTerm2 and on MacOS 10.13.6.
I see there's a .bash_sessions directory with some sessions saved but I guess this is probably an iTerm2 thing to preserve sessions for some reason or another.
terminal bash iterm
terminal bash iterm
asked Mar 25 at 10:33
Dark Star1Dark Star1
249211
249211
unix.stackexchange.com/questions/163371/…
– JBis
Mar 25 at 11:32
add a comment |
unix.stackexchange.com/questions/163371/…
– JBis
Mar 25 at 11:32
unix.stackexchange.com/questions/163371/…
– JBis
Mar 25 at 11:32
unix.stackexchange.com/questions/163371/…
– JBis
Mar 25 at 11:32
add a comment |
1 Answer
1
active
oldest
votes
There can be multiple reasons as to why this happens - I'll try to outline how it works below.
However, I can say that the other answer you have received here is not correct. HISTFILESIZE and HISTSIZE will not cause your history to "sometimes" being entirely wiped out or almost wiped out. Only by setting them to 0 you would get nothing in the files - but it would happen every time, and not by chance. In addition, what you describe with the file being "almost wiped out" cannot happen due to it being set to 0.
You're actually on to the right thing yourself by mentioning .bash_sessions. That is not an iTerm2 "thing", but rather how it works by default on a standard macOS install. Apple has built-in per-session history on top of a regular bash install.
This means that if you have multiple terminals running (for example multiple tabs), each of those will have a seperate history tracked in .bash_sessions. If you reboot your Mac and the terminal windows are restored, you'll find that each still have their own history - and only their own history.
When you close down a bash session, Apple's system will merge the history for that specific session into the global .bash_history file. Then when you open a new terminal (and thus bash session), it will start with that merged history containing history from potentially multiple sessions.
This is all handled by the /etc/bashrc_Apple_Terminal
script.
Now that you know how it works, here are some possible causes for what you're seeing:
You may accidentially have removed the history yourself (history -c)
Your Mac could be infected with malware and/or hacked, and someone else is removing your .bash_history file
You have software installed that periodically empties/removes .bash_history
These reasons are not that likely, I would say.
Now there's one last possible cause:
There's a known race-condition in handling the bash histories. If you close down multiple bash sessions at once (for example closing down a terminal window with multiple tabs), you might loose part of or the whole history.
This happens if the computer is "slow" while reading/writing parts of the history. Basically how it works is that your currently history is read, the file is deleted/moved away, and a new file is created, where all the history lines are written to. If this process is running multiple times at the same time, you might see that one process removes the old history, the other process read an empty history, then the first process writes out the whole history again, and then the other process deletes that and writes out a very short history containing only few recent lines - as it didn't read anything in. This bug shows up as exactly what you have described.
The fix is simply to close one bash session at a time to let the system have time to write out everything correctly.
Thanks for your explanation I will see if I can replicate this given your answer and get back to you. I am aware that it may not necessarily happen on the first try, but I'd like to be sure it is replicable.
– Dark Star1
Mar 25 at 14:18
Also just want to add to the malware part - a common OS X RAT known as Eggshell has a macOS payload that actually executeshistory -wc
as part of its code after the main payload, such that all traces of the payload injection command are wiped. Not saying its necessary Eggshell but it could be something common if someone wanted to hide their traces...
– QuickishFM
Mar 25 at 14:31
This has only happened to me twice and I doubt it is that malware but thanks for the info. Ill look out for it just in case
– Dark Star1
Mar 25 at 15:16
As a workaround, I have this in my bashrc:export PROMPT_COMMAND='echo $(date +%Y-%m-%dT%H:%M:%S) $(pwd): "$(history 1 | head -c 512 | sed "s/^[ ]*[0-9]+[ ]*//" )" >> ~/.bash_history_full; echo -ne "33]0;${PWD##*/}07"'
which will store each command after it is executed.
– jpa
Mar 25 at 19:18
The disadvantage there is thought that it won't seperate sessions for you - so it will be hard to make sense of stuff afterwards if you have many sessions (for example some local and some remote).
– jksoegaard
Mar 25 at 19:42
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
There can be multiple reasons as to why this happens - I'll try to outline how it works below.
However, I can say that the other answer you have received here is not correct. HISTFILESIZE and HISTSIZE will not cause your history to "sometimes" being entirely wiped out or almost wiped out. Only by setting them to 0 you would get nothing in the files - but it would happen every time, and not by chance. In addition, what you describe with the file being "almost wiped out" cannot happen due to it being set to 0.
You're actually on to the right thing yourself by mentioning .bash_sessions. That is not an iTerm2 "thing", but rather how it works by default on a standard macOS install. Apple has built-in per-session history on top of a regular bash install.
This means that if you have multiple terminals running (for example multiple tabs), each of those will have a seperate history tracked in .bash_sessions. If you reboot your Mac and the terminal windows are restored, you'll find that each still have their own history - and only their own history.
When you close down a bash session, Apple's system will merge the history for that specific session into the global .bash_history file. Then when you open a new terminal (and thus bash session), it will start with that merged history containing history from potentially multiple sessions.
This is all handled by the /etc/bashrc_Apple_Terminal
script.
Now that you know how it works, here are some possible causes for what you're seeing:
You may accidentially have removed the history yourself (history -c)
Your Mac could be infected with malware and/or hacked, and someone else is removing your .bash_history file
You have software installed that periodically empties/removes .bash_history
These reasons are not that likely, I would say.
Now there's one last possible cause:
There's a known race-condition in handling the bash histories. If you close down multiple bash sessions at once (for example closing down a terminal window with multiple tabs), you might loose part of or the whole history.
This happens if the computer is "slow" while reading/writing parts of the history. Basically how it works is that your currently history is read, the file is deleted/moved away, and a new file is created, where all the history lines are written to. If this process is running multiple times at the same time, you might see that one process removes the old history, the other process read an empty history, then the first process writes out the whole history again, and then the other process deletes that and writes out a very short history containing only few recent lines - as it didn't read anything in. This bug shows up as exactly what you have described.
The fix is simply to close one bash session at a time to let the system have time to write out everything correctly.
Thanks for your explanation I will see if I can replicate this given your answer and get back to you. I am aware that it may not necessarily happen on the first try, but I'd like to be sure it is replicable.
– Dark Star1
Mar 25 at 14:18
Also just want to add to the malware part - a common OS X RAT known as Eggshell has a macOS payload that actually executeshistory -wc
as part of its code after the main payload, such that all traces of the payload injection command are wiped. Not saying its necessary Eggshell but it could be something common if someone wanted to hide their traces...
– QuickishFM
Mar 25 at 14:31
This has only happened to me twice and I doubt it is that malware but thanks for the info. Ill look out for it just in case
– Dark Star1
Mar 25 at 15:16
As a workaround, I have this in my bashrc:export PROMPT_COMMAND='echo $(date +%Y-%m-%dT%H:%M:%S) $(pwd): "$(history 1 | head -c 512 | sed "s/^[ ]*[0-9]+[ ]*//" )" >> ~/.bash_history_full; echo -ne "33]0;${PWD##*/}07"'
which will store each command after it is executed.
– jpa
Mar 25 at 19:18
The disadvantage there is thought that it won't seperate sessions for you - so it will be hard to make sense of stuff afterwards if you have many sessions (for example some local and some remote).
– jksoegaard
Mar 25 at 19:42
add a comment |
There can be multiple reasons as to why this happens - I'll try to outline how it works below.
However, I can say that the other answer you have received here is not correct. HISTFILESIZE and HISTSIZE will not cause your history to "sometimes" being entirely wiped out or almost wiped out. Only by setting them to 0 you would get nothing in the files - but it would happen every time, and not by chance. In addition, what you describe with the file being "almost wiped out" cannot happen due to it being set to 0.
You're actually on to the right thing yourself by mentioning .bash_sessions. That is not an iTerm2 "thing", but rather how it works by default on a standard macOS install. Apple has built-in per-session history on top of a regular bash install.
This means that if you have multiple terminals running (for example multiple tabs), each of those will have a seperate history tracked in .bash_sessions. If you reboot your Mac and the terminal windows are restored, you'll find that each still have their own history - and only their own history.
When you close down a bash session, Apple's system will merge the history for that specific session into the global .bash_history file. Then when you open a new terminal (and thus bash session), it will start with that merged history containing history from potentially multiple sessions.
This is all handled by the /etc/bashrc_Apple_Terminal
script.
Now that you know how it works, here are some possible causes for what you're seeing:
You may accidentially have removed the history yourself (history -c)
Your Mac could be infected with malware and/or hacked, and someone else is removing your .bash_history file
You have software installed that periodically empties/removes .bash_history
These reasons are not that likely, I would say.
Now there's one last possible cause:
There's a known race-condition in handling the bash histories. If you close down multiple bash sessions at once (for example closing down a terminal window with multiple tabs), you might loose part of or the whole history.
This happens if the computer is "slow" while reading/writing parts of the history. Basically how it works is that your currently history is read, the file is deleted/moved away, and a new file is created, where all the history lines are written to. If this process is running multiple times at the same time, you might see that one process removes the old history, the other process read an empty history, then the first process writes out the whole history again, and then the other process deletes that and writes out a very short history containing only few recent lines - as it didn't read anything in. This bug shows up as exactly what you have described.
The fix is simply to close one bash session at a time to let the system have time to write out everything correctly.
Thanks for your explanation I will see if I can replicate this given your answer and get back to you. I am aware that it may not necessarily happen on the first try, but I'd like to be sure it is replicable.
– Dark Star1
Mar 25 at 14:18
Also just want to add to the malware part - a common OS X RAT known as Eggshell has a macOS payload that actually executeshistory -wc
as part of its code after the main payload, such that all traces of the payload injection command are wiped. Not saying its necessary Eggshell but it could be something common if someone wanted to hide their traces...
– QuickishFM
Mar 25 at 14:31
This has only happened to me twice and I doubt it is that malware but thanks for the info. Ill look out for it just in case
– Dark Star1
Mar 25 at 15:16
As a workaround, I have this in my bashrc:export PROMPT_COMMAND='echo $(date +%Y-%m-%dT%H:%M:%S) $(pwd): "$(history 1 | head -c 512 | sed "s/^[ ]*[0-9]+[ ]*//" )" >> ~/.bash_history_full; echo -ne "33]0;${PWD##*/}07"'
which will store each command after it is executed.
– jpa
Mar 25 at 19:18
The disadvantage there is thought that it won't seperate sessions for you - so it will be hard to make sense of stuff afterwards if you have many sessions (for example some local and some remote).
– jksoegaard
Mar 25 at 19:42
add a comment |
There can be multiple reasons as to why this happens - I'll try to outline how it works below.
However, I can say that the other answer you have received here is not correct. HISTFILESIZE and HISTSIZE will not cause your history to "sometimes" being entirely wiped out or almost wiped out. Only by setting them to 0 you would get nothing in the files - but it would happen every time, and not by chance. In addition, what you describe with the file being "almost wiped out" cannot happen due to it being set to 0.
You're actually on to the right thing yourself by mentioning .bash_sessions. That is not an iTerm2 "thing", but rather how it works by default on a standard macOS install. Apple has built-in per-session history on top of a regular bash install.
This means that if you have multiple terminals running (for example multiple tabs), each of those will have a seperate history tracked in .bash_sessions. If you reboot your Mac and the terminal windows are restored, you'll find that each still have their own history - and only their own history.
When you close down a bash session, Apple's system will merge the history for that specific session into the global .bash_history file. Then when you open a new terminal (and thus bash session), it will start with that merged history containing history from potentially multiple sessions.
This is all handled by the /etc/bashrc_Apple_Terminal
script.
Now that you know how it works, here are some possible causes for what you're seeing:
You may accidentially have removed the history yourself (history -c)
Your Mac could be infected with malware and/or hacked, and someone else is removing your .bash_history file
You have software installed that periodically empties/removes .bash_history
These reasons are not that likely, I would say.
Now there's one last possible cause:
There's a known race-condition in handling the bash histories. If you close down multiple bash sessions at once (for example closing down a terminal window with multiple tabs), you might loose part of or the whole history.
This happens if the computer is "slow" while reading/writing parts of the history. Basically how it works is that your currently history is read, the file is deleted/moved away, and a new file is created, where all the history lines are written to. If this process is running multiple times at the same time, you might see that one process removes the old history, the other process read an empty history, then the first process writes out the whole history again, and then the other process deletes that and writes out a very short history containing only few recent lines - as it didn't read anything in. This bug shows up as exactly what you have described.
The fix is simply to close one bash session at a time to let the system have time to write out everything correctly.
There can be multiple reasons as to why this happens - I'll try to outline how it works below.
However, I can say that the other answer you have received here is not correct. HISTFILESIZE and HISTSIZE will not cause your history to "sometimes" being entirely wiped out or almost wiped out. Only by setting them to 0 you would get nothing in the files - but it would happen every time, and not by chance. In addition, what you describe with the file being "almost wiped out" cannot happen due to it being set to 0.
You're actually on to the right thing yourself by mentioning .bash_sessions. That is not an iTerm2 "thing", but rather how it works by default on a standard macOS install. Apple has built-in per-session history on top of a regular bash install.
This means that if you have multiple terminals running (for example multiple tabs), each of those will have a seperate history tracked in .bash_sessions. If you reboot your Mac and the terminal windows are restored, you'll find that each still have their own history - and only their own history.
When you close down a bash session, Apple's system will merge the history for that specific session into the global .bash_history file. Then when you open a new terminal (and thus bash session), it will start with that merged history containing history from potentially multiple sessions.
This is all handled by the /etc/bashrc_Apple_Terminal
script.
Now that you know how it works, here are some possible causes for what you're seeing:
You may accidentially have removed the history yourself (history -c)
Your Mac could be infected with malware and/or hacked, and someone else is removing your .bash_history file
You have software installed that periodically empties/removes .bash_history
These reasons are not that likely, I would say.
Now there's one last possible cause:
There's a known race-condition in handling the bash histories. If you close down multiple bash sessions at once (for example closing down a terminal window with multiple tabs), you might loose part of or the whole history.
This happens if the computer is "slow" while reading/writing parts of the history. Basically how it works is that your currently history is read, the file is deleted/moved away, and a new file is created, where all the history lines are written to. If this process is running multiple times at the same time, you might see that one process removes the old history, the other process read an empty history, then the first process writes out the whole history again, and then the other process deletes that and writes out a very short history containing only few recent lines - as it didn't read anything in. This bug shows up as exactly what you have described.
The fix is simply to close one bash session at a time to let the system have time to write out everything correctly.
answered Mar 25 at 13:14
jksoegaardjksoegaard
20.2k2150
20.2k2150
Thanks for your explanation I will see if I can replicate this given your answer and get back to you. I am aware that it may not necessarily happen on the first try, but I'd like to be sure it is replicable.
– Dark Star1
Mar 25 at 14:18
Also just want to add to the malware part - a common OS X RAT known as Eggshell has a macOS payload that actually executeshistory -wc
as part of its code after the main payload, such that all traces of the payload injection command are wiped. Not saying its necessary Eggshell but it could be something common if someone wanted to hide their traces...
– QuickishFM
Mar 25 at 14:31
This has only happened to me twice and I doubt it is that malware but thanks for the info. Ill look out for it just in case
– Dark Star1
Mar 25 at 15:16
As a workaround, I have this in my bashrc:export PROMPT_COMMAND='echo $(date +%Y-%m-%dT%H:%M:%S) $(pwd): "$(history 1 | head -c 512 | sed "s/^[ ]*[0-9]+[ ]*//" )" >> ~/.bash_history_full; echo -ne "33]0;${PWD##*/}07"'
which will store each command after it is executed.
– jpa
Mar 25 at 19:18
The disadvantage there is thought that it won't seperate sessions for you - so it will be hard to make sense of stuff afterwards if you have many sessions (for example some local and some remote).
– jksoegaard
Mar 25 at 19:42
add a comment |
Thanks for your explanation I will see if I can replicate this given your answer and get back to you. I am aware that it may not necessarily happen on the first try, but I'd like to be sure it is replicable.
– Dark Star1
Mar 25 at 14:18
Also just want to add to the malware part - a common OS X RAT known as Eggshell has a macOS payload that actually executeshistory -wc
as part of its code after the main payload, such that all traces of the payload injection command are wiped. Not saying its necessary Eggshell but it could be something common if someone wanted to hide their traces...
– QuickishFM
Mar 25 at 14:31
This has only happened to me twice and I doubt it is that malware but thanks for the info. Ill look out for it just in case
– Dark Star1
Mar 25 at 15:16
As a workaround, I have this in my bashrc:export PROMPT_COMMAND='echo $(date +%Y-%m-%dT%H:%M:%S) $(pwd): "$(history 1 | head -c 512 | sed "s/^[ ]*[0-9]+[ ]*//" )" >> ~/.bash_history_full; echo -ne "33]0;${PWD##*/}07"'
which will store each command after it is executed.
– jpa
Mar 25 at 19:18
The disadvantage there is thought that it won't seperate sessions for you - so it will be hard to make sense of stuff afterwards if you have many sessions (for example some local and some remote).
– jksoegaard
Mar 25 at 19:42
Thanks for your explanation I will see if I can replicate this given your answer and get back to you. I am aware that it may not necessarily happen on the first try, but I'd like to be sure it is replicable.
– Dark Star1
Mar 25 at 14:18
Thanks for your explanation I will see if I can replicate this given your answer and get back to you. I am aware that it may not necessarily happen on the first try, but I'd like to be sure it is replicable.
– Dark Star1
Mar 25 at 14:18
Also just want to add to the malware part - a common OS X RAT known as Eggshell has a macOS payload that actually executes
history -wc
as part of its code after the main payload, such that all traces of the payload injection command are wiped. Not saying its necessary Eggshell but it could be something common if someone wanted to hide their traces...– QuickishFM
Mar 25 at 14:31
Also just want to add to the malware part - a common OS X RAT known as Eggshell has a macOS payload that actually executes
history -wc
as part of its code after the main payload, such that all traces of the payload injection command are wiped. Not saying its necessary Eggshell but it could be something common if someone wanted to hide their traces...– QuickishFM
Mar 25 at 14:31
This has only happened to me twice and I doubt it is that malware but thanks for the info. Ill look out for it just in case
– Dark Star1
Mar 25 at 15:16
This has only happened to me twice and I doubt it is that malware but thanks for the info. Ill look out for it just in case
– Dark Star1
Mar 25 at 15:16
As a workaround, I have this in my bashrc:
export PROMPT_COMMAND='echo $(date +%Y-%m-%dT%H:%M:%S) $(pwd): "$(history 1 | head -c 512 | sed "s/^[ ]*[0-9]+[ ]*//" )" >> ~/.bash_history_full; echo -ne "33]0;${PWD##*/}07"'
which will store each command after it is executed.– jpa
Mar 25 at 19:18
As a workaround, I have this in my bashrc:
export PROMPT_COMMAND='echo $(date +%Y-%m-%dT%H:%M:%S) $(pwd): "$(history 1 | head -c 512 | sed "s/^[ ]*[0-9]+[ ]*//" )" >> ~/.bash_history_full; echo -ne "33]0;${PWD##*/}07"'
which will store each command after it is executed.– jpa
Mar 25 at 19:18
The disadvantage there is thought that it won't seperate sessions for you - so it will be hard to make sense of stuff afterwards if you have many sessions (for example some local and some remote).
– jksoegaard
Mar 25 at 19:42
The disadvantage there is thought that it won't seperate sessions for you - so it will be hard to make sense of stuff afterwards if you have many sessions (for example some local and some remote).
– jksoegaard
Mar 25 at 19:42
add a comment |
unix.stackexchange.com/questions/163371/…
– JBis
Mar 25 at 11:32