Bash eternal history

Posted by ateijelo on Wed 15 Aug 2007 at 10:08

Tags: , ,

Many times I've found myself using Ctrl-R in Bash to get a old command four times the terminal width, just to find out that too many days have passed and it's no longer in the .bash_history file. Here are two lines that will keep track of every command line you type at the bash prompt and use no external processes at all, just plain bash.

My first approach to this problem was increasing the maximum number of lines in the history to a very large quantity. But no matter how large it was, there was always a moment when I needed a long command I typed many months ago and it had already left the history. The current solution came to my mind when I learned about the PROMPT_COMMAND variable, a command that bash executes before showing each prompt. Here are the two lines:

export HISTTIMEFORMAT="%s "
PROMPT_COMMAND="${PROMPT_COMMAND:+$PROMPT_COMMAND ; }"'echo $$ $USER \
               "$(history 1)" >> ~/.bash_eternal_history'

One goal I set to myself was to achieve it without using any external process, so bash wouldn't have to fork a new process after every ENTER pressed at its prompt. The first line sets the format of history lines to include the date as a Unix timestamp, so you can now when you typed every command. The second line, which is the core of the solution, first ensures that, if a previous PROMPT_COMMAND was set, it gets executed before our stuff and then appends a line of the format:

PID USER INDEX TIMESTAMP COMMAND

to a file called .bash_eternal_history in the current user home.

Adding the username, which at first seemed unnecesary, became useful later to distiguish between "sudo -s" sessions and normal sessions which retain the same value for "~/", and so append lines to the same .bash_eternal_history file.

I hope some of you find these two lines as useful as I do. :-)

 

 


Posted by TokenGoth (193.195.xx.xx) on Wed 15 Aug 2007 at 12:25
On a similar note - is there a way to record/audit every command entered by any user?

[ Parent | Reply to this comment ]

Posted by Anonymous (83.24.xx.xx) on Wed 15 Aug 2007 at 13:04
Use snoopy.

[ Parent | Reply to this comment ]

Posted by Steve (80.68.xx.xx) on Wed 15 Aug 2007 at 14:20
[ View Steve's Scratchpad | View Weblogs ]

Snoopy does work well, and we've previously introduced it here.

Steve

[ Parent | Reply to this comment ]

Posted by Anonymous (71.98.xx.xx) on Wed 15 Aug 2007 at 13:12
This sounds nice and cool... until one day you do something like "mysql -u mysqluser -p mysqlpassword" - and that string goes directly to your eternal history file. Where I will be very happy to read it, because you don't mention anything about securing it from the start (and thus it'll have world-readable 0644 permissions by default). This .bash_eternal_history file must be chmod-ed to 0600. Just like the regular .bash_history is ;-) Lev

[ Parent | Reply to this comment ]

Posted by davux (82.226.xx.xx) on Wed 15 Aug 2007 at 13:24

Just as a sidenote: the "-p" parameter of mysql is not followed by the password, it's just there to ask the password to be prompted interactively.

Your remark is still valid for other programs, though (like ncftp -p password or ldapsearch -w password).

[ Parent | Reply to this comment ]

Posted by chris (82.196.xx.xx) on Wed 15 Aug 2007 at 13:47
[ View Weblogs ]
mysql will take the password on the command line if you specify -pyourpassword - note - no space between -p and the password

[ Parent | Reply to this comment ]

Posted by Anonymous (213.164.xx.xx) on Fri 17 Aug 2007 at 10:30
or --password

[ Parent | Reply to this comment ]

Posted by Anonymous (81.170.xx.xx) on Fri 17 Aug 2007 at 19:32
granted putting it in a file is probably even worse, but be aware that such a password on the command-line will often be visible with ps.

in general, not a good idea to write a password on the command-line like this.

[ Parent | Reply to this comment ]

Posted by Anonymous (86.53.xx.xx) on Wed 15 Aug 2007 at 13:43
Very nice. This also gets around the "I did it in terminal X but terminal Y's history was saved instead" problem.

Just one problem to be aware of... when you first log in, there will be a random entry written to the file based on the the previous command you wrote that was saved to history in your last session.

thanks!

[ Parent | Reply to this comment ]

Posted by Anonymous (200.55.xx.xx) on Wed 15 Aug 2007 at 18:27
Indeed. And also, each time you hit ENTER without a command, the last non-empty command is appended again. And there's no way to distinguish between actually having typed <UP><ENTER> or just having typed <ENTER>. I haven't found a way to solve those two problem while still keeping it simple. Maybe implementing it all in a function.

[ Parent | Reply to this comment ]

Posted by scheuref (46.14.xx.xx) on Tue 26 Mar 2013 at 13:32
this issue is solved by using traps.
see http://blog.pointsoftware.ch/index.php/howto-bash-audit-command-l ogger

francois

[ Parent | Reply to this comment ]

Posted by drgraefy (128.59.xx.xx) on Wed 15 Aug 2007 at 19:45
[ View Weblogs ]
It seems to me you can achieve the same affect by using the "histappend" bash shell option. From the bash man page, under the HISTORY section:
On startup, the history is initialized from the file named by  the  varia-
able  HISTFILE (default ~/.bash_history).  The file named by the value of
HISTFILE is truncated, if necessary, to contain no more than  the  number
of  lines  specified  by  the value of HISTFILESIZE.  When an interactive
shell exits, the last $HISTSIZE lines are copied from the history list to
$HISTFILE.   If  the histappend shell option is enabled (see the descrip-
tion of shopt under SHELL BUILTIN COMMANDS below), the lines are appended
to the history file, otherwise the history file is overwritten.  If HIST-
FILE is unset, or if the history file is unwritable, the history  is  not
saved.   After  saving the history, the history file is truncated to con-
tain no more than HISTFILESIZE lines.  If HISTFILESIZE  is  not  set,  no
truncation is performed.
I haven't actually started using it yet (although I plan to), but I think you enable it by just adding
shopt -s histappend
to your .bashrc.

[ Parent | Reply to this comment ]

Posted by Anonymous (68.49.xx.xx) on Sat 29 Sep 2012 at 12:01
The difference is that one doesn't keep them all in bash's actual "history" [and therefore, doesn't cause bash to load an arbitrarily large file every time you start a shell, eventually causing hilarious memory/processor usage].

[ Parent | Reply to this comment ]

Posted by Anonymous (84.221.xx.xx) on Fri 17 Aug 2007 at 11:34
This functionality is already available in Bash.
If you unset HISTSIZE, every Bash session keeps in memory all the commands entered into it (instead of the last $HISTSIZE only.) If you unset HISTFILESIZE, at the end of each session Bash inconditionally appends all the commands it has in memory to the history file (instead of truncating the file to be at most $HISTFILESIZE lines.)
I've been using the following in my .bashrc for a long time:

unset HISTSIZE
unset HISTFILESIZE

Granted, it does'n have that fancy "PID USER INDEX TIMESTAMP" thing, but it's simple, builtin and fast.

[ Parent | Reply to this comment ]

Posted by yarikoptic (165.230.xx.xx) on Sat 18 Aug 2007 at 05:18
> If you unset HISTSIZE, every Bash session keeps in memory all the commands entered into it

And that would be a problem for me
$> wc .bash_history.archive
1871262 5533327 45226246 .bash_history.archive
I wonder how bash would perform... lets see...
koshka@ravana:~$ date; bash
Sat Aug 18 00:13:50 EDT 2007
date
koshka@ravana:~$ date
Sat Aug 18 00:13:56 EDT 2007
koshka@ravana:~$ ps u -p $$
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
koshka 7029 36.8 4.1 348524 337784 pts/110 S 00:13 0:05 bash

So it took it 6 seconds to start up (on a quite good box) and it took quite a good chunk of resident memory: usually I have smth like
$> ps u -p $$
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
yoh 5188 0.0 0.0 12928 4972 pts/110 Ss 00:06 0:00 bash
with 700 or so history lines.

It can be partially cured by (semi)automated periodic dump of it to another file and its tailing into the "working set".

About alternative approach I did take you can read in elderly post on d-a
http://www.debian-administration.org/articles/175

Advantage imho over suggested method is that all commands in the same session are dumped together, wherewere (if I got it right) if I have multiple bashes running, resultant eternal history will have interleaved commands from different sessions and additional postprocessing would be necessary to select a needed one.

[ Parent | Reply to this comment ]

Posted by hq4ever (87.69.xx.xx) on Sat 22 Sep 2007 at 17:47
A possible solution could be logrotate to rotate the .bash_history, with not upper limit of logs. Move it to a subfolder, such as .bash_history.past and do not gzip it. I suggest rotating it on a size based criteria (>10MB).


You would lose the Ctrl + R but still will be able to grep for you desired command.

[ Parent | Reply to this comment ]

Posted by vegiVamp (195.177.xx.xx) on Mon 20 Aug 2007 at 14:37
Still pretty pointless if you've got oodles of machines to manage, and can't remember where you last executed 'that command that did something like foo and might have included bar and/or baz somewhere'.

I simply track stuff like this in my personal documentation, which I keep in TiddlyWiki.

For those who don't know it, it's a Wiki (doh), but doesn't require anything but an ajax-capable browser - it's a single HTML file with JavaScript and whatnot, doesn't even require a webserver. I just keep it on my USB stick, available wherever I am :-)

[ Parent | Reply to this comment ]

Posted by yarikoptic (24.152.xx.xx) on Tue 21 Aug 2007 at 02:21
TiddlyWiki is indeed a very nice note keeping tool. +1 on that. Especially since it is version control friendly (I have elderly CVS on top of it), so you can keep it in sync on different boxes.
But eternal bash history solves other problems. In my implementation, which I mentioned before, I also titled every dump of history with a hostname, so on home directory shared across different boxes I have knowledge where it was ran. Also you can easily set it up to dump to separate files for each box and keep those files in sync via VCS or smth like sync/unison.

[ Parent | Reply to this comment ]

Posted by Anonymous (2002:0xx:0xx:0xxx:0xxx:0xxx:xx) on Thu 11 Oct 2007 at 16:47
It may not suit everyone, but a technique that I find extremely useful is to keep a separate history for each directory. Generally I find that my commands are very context-sensitive -- when I'm in project X's folder, I'm trying to run one set of commands, and when I'm in ~/etc, I'm doing other sorts of things. And generally when I'm trying to remember some arcana that I invoked ages ago, it's in a particular context like that.

So I use the following bash function:
#
# Usage: mycd <path>
#
#  Replacement for builtin 'cd', wh ich keeps a separate bash-history
#   for every directory.
function mycd()
{
history -w # write current history  file
builtin cd "$@"  # do actual c d
local HISTDIR="$HOME/.dir_bash_history$PWD" # use& nbsp;nested folders for history
if  [ ! -d "$HISTDIR" ]&n bsp;; then # create folder if neede d
mkdir -p "$HISTDIR"
fi
export HISTFILE="$HISTDIR/bash_history.txt" # set& nbsp;new history file
history -c  # clear memory
history -r #read from current histfile
}


and then set it up with the following in my bashrc:

shopt -s histappend
alias cd="mycd"
export HISTFILE="$HOME/.dir_bash_history$PWD/bash_history.tx t"


[ Parent | Reply to this comment ]

Posted by Anonymous (65.200.xx.xx) on Tue 20 Apr 2010 at 15:12
I do something similar with a .bashrc that contains:

HISTSIZE=512
HISTFILESIZE=40960
HISTCONTROL=ignoredups
HISTIGNORE=?:??
PROMPT_COMMAND='history -a'


HISTSIZE=512

Keep a reasonable amount of history in memory. Preloaded from the file when bash starts, then appended with commands from this session. To search back farther I grep .bash_history

HISTFILESIZE=40960

Set the size large enough to hold a couple years worth of commands, but not infinite.

HISTCONTROL=ignoredups

Don't clutter the file with consecutively repeated commands

HISTIGNORE=?:??

Don't clutter the file with trivial one and two character commands

PROMPT_COMMAND='history -a'

Append the latest command to the file at each prompt.

[ Parent | Reply to this comment ]

Posted by Anonymous (71.207.xx.xx) on Mon 14 Mar 2011 at 02:39
I also remove non-consecutive duplicate entries with this function:

dupd ()
{
[ -z "$1" ] && return;
awk ' !x[$0]++' $1 > temp;
cp temp $1
}

which I add to my .bashrc:

banner "Remove duplicate entries in $HISTFILE"
wc -l $HISTFILE
dupd $HISTFILE
wc -l $HISTFILE

[ Parent | Reply to this comment ]

Posted by Anonymous (64.102.xx.xx) on Wed 30 Mar 2011 at 21:04
anyway to rotate the log file if it gets too big?

[ Parent | Reply to this comment ]

Posted by ateijelo (200.55.xx.xx) on Thu 31 Mar 2011 at 00:14
Well, I've never really thought of rotating the file. But I can tell you from my experience. My first eternal history has 1169750159 as the timestamp of the first line, which is January 25th 2007, more than 4 years ago; it has more than 128000 lines and it weighs a bit less than 7Mb. I still can grep it quickly and appending still isn't noticeable. Of course, that's just my usage pattern. Some people will surely use the command line more.

But you're right, eventually it might get too heavy. Maybe logrotate could be used to rotate it, back it up, etc.

[ Parent | Reply to this comment ]

Posted by CDSRV (70.183.xx.xx) on Sat 7 Apr 2012 at 15:30
>> "maybe logrotate" .. good point, especially the 'dateext' feature that versions the file, but it seems like using cron to gather these details leaves too much time for data to show up missing.. what happens when you need to find out who issued the last command right before the server crashed? waiting for cron to run certainly won't help 'then'..

in general, this type of naming can let multiple files from multiple users and multiple hosts all co-exist in the same namespace..

$YYMMDD-HHSS-$LOGNAME-$HOSTNAME.ext

where,

$HISTFILE=$(tee ~/my$HISTFILE > /var/log/bash/$STAMPED_HISTFILE)

(( * ^^ just the basic idea here -- untested & likely-incorrect syntax ^^ * ;)


http://sock-raw.org/papers/bash_history

^^ in this this article discussing 'local0.crit' logging to hist.log, -- it doesn't seem to work on a curent out-the-box ubuntu setup(yes, even after restarting the syslogd) ..this new feature 'doesn't differentiate' between users?.. anyone got this working??

beyond all of the valid limitations cited, there really is a wide range of legitimate, practical uses for this, and it might actually help mitigate some of those risks for all but the most sophisticated adversary..

basically, any servers with multiple active shell accounts could benefit from simple logging that shows who, what, when, and where.. in combination with other types of logging, this could provide a decent amount of useful and practical reporting.

process accounting is of course very thorough, but quite a bit more complex and generates tons of data.


http://www.itss.it/documentazione/tecniche/sicurezza_di_base/Log_ every_User_Command.html

^^ this seems useful but it depends on sudo and apparently does not cover the root user, arguably the most important account to log::



http://www.egbok.com/sudoscript/sudoshell.1.html

^^ this is another approach.. could be updated::


--------------------------------------------------

specs for a good solution:

- minimal external dependencies (for so many reasons)
- log session identifiers to configurable filename and/or in the file-headers
- log session commands to two or more files at a time (>>pipe|tee <<$append)
- include the configs for all relevant custom shell variables and settings (pick and choose, rather than research and figure out)
- reliably record the session activities, either as the commands are executed or on exit.
- provide a hook/trigger for filters, notifications or more complex rotate/merge/sync/copy/purge functions


:)

[ Parent | Reply to this comment ]

Posted by CDSRV (70.183.xx.xx) on Sat 7 Apr 2012 at 15:48
and, snoopy looks nice, but would it easily fill up a partition quickly by logging every single 'execve' system call on the entire system, just like process accounting..??

obviously, not logging the script contents/actions is a big issue which can only be addressed by some lower level auditing function.. but the niche here is for "login sessions", "commands typed" , etc.

:)

[ Parent | Reply to this comment ]

Posted by Anonymous (204.244.xx.xx) on Mon 16 Apr 2012 at 21:02
This is really not the way to do it. There is a patch for bash 3.2 that is working and will probably be updated to work for newer versions. If bash is the one to do the logging than many problems are actually solved.

Also the "insecure" part is not actually true or relevant as removing info from bash history or NOT keeping logs of activity is never a solution to a security problem. Logs are a separate pretty important and extensive part of system/network security and this is just another log file that should be treated correctly to avoid exposing critical info.

Also, starting with bash 4.1 there is a built-in feature to create a log file with all the commands bash is actually processing and the info from this file correlated with other log message can be very helpful in tracing user activity.
However, in order to make it safe and impossible to disable by the users, there no switch to toggle this feature and because it comes commented in the source code you have to remove the comments and rebuild the rpm or recompile bash.

Also, for security reasons, have only ONE shell available on a system at all times so users cannot switch or get different shells assigned.

[ Parent | Reply to this comment ]

Sign In

Username:

Password:

[Register|Advanced]

 

Flattr

 

Current Poll

What do you use for configuration management?








( 143 votes ~ 0 comments )