Bash eternal history
Posted by ateijelo on Wed 15 Aug 2007 at 10:08
Many times I've found myself using Ctrl-R in Bash to get a old command four times the terminal width, just to find out that too many days have passed and it's no longer in the .bash_history file. Here are two lines that will keep track of every command line you type at the bash prompt and use no external processes at all, just plain bash.
My first approach to this problem was increasing the maximum number of lines in the history to a very large quantity. But no matter how large it was, there was always a moment when I needed a long command I typed many months ago and it had already left the history. The current solution came to my mind when I learned about the PROMPT_COMMAND variable, a command that bash executes before showing each prompt. Here are the two lines:
export HISTTIMEFORMAT="%s "
PROMPT_COMMAND="${PROMPT_COMMAND:+$PROMPT_COMMAND ; }"'echo $$ $USER \
"$(history 1)" >> ~/.bash_eternal_history'
One goal I set to myself was to achieve it without using any external process, so bash wouldn't have to fork a new process after every ENTER pressed at its prompt. The first line sets the format of history lines to include the date as a Unix timestamp, so you can now when you typed every command. The second line, which is the core of the solution, first ensures that, if a previous PROMPT_COMMAND was set, it gets executed before our stuff and then appends a line of the format:
PID USER INDEX TIMESTAMP COMMAND
to a file called .bash_eternal_history in the current user home.
Adding the username, which at first seemed unnecesary, became useful later to distiguish between "sudo -s" sessions and normal sessions which retain the same value for "~/", and so append lines to the same .bash_eternal_history file.
I hope some of you find these two lines as useful as I do. :-)
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
Just as a sidenote: the "-p" parameter of mysql is not followed by the password, it's just there to ask the password to be prompted interactively.
Your remark is still valid for other programs, though (like ncftp -p password or ldapsearch -w password).
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
in general, not a good idea to write a password on the command-line like this.
[ Parent | Reply to this comment ]
Just one problem to be aware of... when you first log in, there will be a random entry written to the file based on the the previous command you wrote that was saved to history in your last session.
thanks!
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
On startup, the history is initialized from the file named by the varia- able HISTFILE (default ~/.bash_history). The file named by the value of HISTFILE is truncated, if necessary, to contain no more than the number of lines specified by the value of HISTFILESIZE. When an interactive shell exits, the last $HISTSIZE lines are copied from the history list to $HISTFILE. If the histappend shell option is enabled (see the descrip- tion of shopt under SHELL BUILTIN COMMANDS below), the lines are appended to the history file, otherwise the history file is overwritten. If HIST- FILE is unset, or if the history file is unwritable, the history is not saved. After saving the history, the history file is truncated to con- tain no more than HISTFILESIZE lines. If HISTFILESIZE is not set, no truncation is performed.I haven't actually started using it yet (although I plan to), but I think you enable it by just adding
shopt -s histappendto your .bashrc.
[ Parent | Reply to this comment ]
If you unset HISTSIZE, every Bash session keeps in memory all the commands entered into it (instead of the last $HISTSIZE only.) If you unset HISTFILESIZE, at the end of each session Bash inconditionally appends all the commands it has in memory to the history file (instead of truncating the file to be at most $HISTFILESIZE lines.)
I've been using the following in my .bashrc for a long time:
unset HISTSIZE
unset HISTFILESIZE
Granted, it does'n have that fancy "PID USER INDEX TIMESTAMP" thing, but it's simple, builtin and fast.
[ Parent | Reply to this comment ]
And that would be a problem for me
$> wc .bash_history.archive
1871262 5533327 45226246 .bash_history.archive
I wonder how bash would perform... lets see...
koshka@ravana:~$ date; bash
Sat Aug 18 00:13:50 EDT 2007
date
koshka@ravana:~$ date
Sat Aug 18 00:13:56 EDT 2007
koshka@ravana:~$ ps u -p $$
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
koshka 7029 36.8 4.1 348524 337784 pts/110 S 00:13 0:05 bash
So it took it 6 seconds to start up (on a quite good box) and it took quite a good chunk of resident memory: usually I have smth like
$> ps u -p $$
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
yoh 5188 0.0 0.0 12928 4972 pts/110 Ss 00:06 0:00 bash
with 700 or so history lines.
It can be partially cured by (semi)automated periodic dump of it to another file and its tailing into the "working set".
About alternative approach I did take you can read in elderly post on d-a
http://www.debian-administration.org/articles/175
Advantage imho over suggested method is that all commands in the same session are dumped together, wherewere (if I got it right) if I have multiple bashes running, resultant eternal history will have interleaved commands from different sessions and additional postprocessing would be necessary to select a needed one.
[ Parent | Reply to this comment ]
You would lose the Ctrl + R but still will be able to grep for you desired command.
[ Parent | Reply to this comment ]
I simply track stuff like this in my personal documentation, which I keep in TiddlyWiki.
For those who don't know it, it's a Wiki (doh), but doesn't require anything but an ajax-capable browser - it's a single HTML file with JavaScript and whatnot, doesn't even require a webserver. I just keep it on my USB stick, available wherever I am :-)
[ Parent | Reply to this comment ]
But eternal bash history solves other problems. In my implementation, which I mentioned before, I also titled every dump of history with a hostname, so on home directory shared across different boxes I have knowledge where it was ran. Also you can easily set it up to dump to separate files for each box and keep those files in sync via VCS or smth like sync/unison.
[ Parent | Reply to this comment ]
So I use the following bash function:
#
# Usage: mycd <path>
#
# Replacement for builtin 'cd', wh ich keeps a separate bash-history
# for every directory.
function mycd()
{
history -w # write current history file
builtin cd "$@" # do actual c d
local HISTDIR="$HOME/.dir_bash_history$PWD" # use& nbsp;nested folders for history
if [ ! -d "$HISTDIR" ]&n bsp;; then # create folder if neede d
mkdir -p "$HISTDIR"
fi
export HISTFILE="$HISTDIR/bash_history.txt" # set& nbsp;new history file
history -c # clear memory
history -r #read from current histfile
}
and then set it up with the following in my bashrc:
shopt -s histappend
alias cd="mycd"
export HISTFILE="$HOME/.dir_bash_history$PWD/bash_history.tx t"
[ Parent | Reply to this comment ]