New User? Register here - Existing Users: Username: Password: [Advanced Login]

 

 

Current Poll

Your preferred Interactive shell?









( 1341 votes ~ 14 comments )

 

BASH history forever.

Posted by yarikoptic on Sat 2 Jul 2005 at 22:30

Tags: ,

I would like to keep track of what, when and where I've done something in the shell for the rest of my Linux life. It is a reasonable wish to have all of my activites logged, so in the future I could check what I did, how I did, and when I did it. Of cause it imposes some security hazard if you type in your password by mistake while working in the shell prompt, so you should be carefull and have right permissions setup.

This article is excerpt from its original page where you can see how it evolved ;-)

The first solution would be to set HISTSIZE to be very big, but then I don't know how well your bash would behave - I believe it tries to keep them all in memory. But I want to have my bash fast and lightweighted! So it must be accomplished in another way.

Lets use a big file ~/.bash_history.archive (separate from HISTFILE=~/.bash_history). And then on exit from each bash session lets append new history lines to it. To accomplish that we need to remember how far in the history we were at the beginning of the session and at the end of session, so we could dump lines inbetween to our log file. The problem I've ran into is that it is impossible at the time of run of .bash{rc,_profile} to know status of history, thus I ended up scanning HISTFILE, thus I still can have some bugs becuase this way is unnatural.

To enable such historing you need merely to source a .bashrc_history from your ~/.bashrc. Optionally you can modify .inputrc to have shortcut to dump history without exiting shell. I provide source of the script directly in the article so it is available even if my website goes down (I hope it will not in the nearest future)

#!/bin/bash
#-------------------------- =+- Shell script -+= --------------------------
# @file      .bashrc_history.sh
# @date      Thu Mar 10 14:02:36 2005
# @brief
#
# CVS version control block - do not edit manually
#  $RCSfile: .bashrc_history,v $
#  $Source: /home/cvs/yoh/.bashrc_history,v $
#
# Created: Thu Mar 10 14:02:36 2005
#  Commited: $Date: 2005/03/24 14:24:28 $
#  Revision: $Revision: 1.7 $
#
#  Yaroslav Halchenko                                      CS@UNM, CS@NJIT
#  web:     http://www.onerussian.com                      & PSYCH@RUTGERS
#  e-mail:  yoh@onerussian.com                              ICQ#: 60653192
#
# DESCRIPTION (NOTES):
#   A script to be sourced from .bashrc to provide ways to archive all the
#   actions in infinitely long history file.
#   
#   To use, just place
#     'source .bashrc_history'   in ~/.bashrc
#
#     '$if Bash
#      # to exit through calling exit function which will archive the history
#      # next ones are optional: first is left for historical reasons
#      "\C-x\C-x": "exit\n"
#      "\C-x\C-w": "archive_history\n"
#      $endif'                   in ~/.inputrc
#
#   Then whenever you close bash (exit,logout or exit by pressing Ctrl-D
#   or Ctrl-X twice you will have a piece of current history added to
#   ~/.bash_history.archive
#
# SOURCE:
#   http://www.onerussian.com/Linux/.files/.bashrc_history
#
# LICENSE:  
#   Released under GNU Generic Public License. You should've received it with 
#   your GNU/Linux system. If not, write to the Free Software Foundation, Inc.,
#   59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#-----------------\____________________________________/------------------


if [ "$PS1" ] ; then # interactive shell
    export  \
	CURBASHSTART=$(grep -v -e "^[ \t]*$" -e "^\s*#" $HISTFILE 2>/dev/null /dev/null) \
	CURBASHDATE=$(date > $HISTORYOLD
	    history $(($HISTCMD-$CURBASHSTART-1)) | sed -e 's/^[ ]*[0-9][0-9]* [ ]*//g'  >> $HISTORYOLD
	    CURBASHSTART=$(($HISTCMD-1))
	fi
    }

    trap 'archive_history' EXIT

fi

P.S. There might be much better/safer way to reach the goal -- thus comments are very welcome

Share/Save/Bookmark


Posted by tuxy (66.173.xx.xx) on Mon 4 Jul 2005 at 16:05
[ Send Message ]
I tend to setup script to sync it for me:
#!/bin/bash
cd ~/
tail -6000 .bash_history.all | diff .bash_history - | sed -n 's/^< //p' >> .bash_history.all
and then run it nightly in cron.

[ Parent | Reply to this comment ]

Posted by yarikoptic (4.64.xx.xx) on Mon 4 Jul 2005 at 20:10
[ Send Message ]
seems to be a much nicer way than mine :-). Although it doesn't have separation between different "sessions" it should work for my purposes unless my terminal session gets too long -- sometimes I work in the same terminal session for months, that is why I like to archive_history from time to time to don't lose "valuable" logs if system crashes.

[ Parent | Reply to this comment ]

Posted by Anonymous (24.44.xx.xx) on Thu 28 Jul 2005 at 06:16
I was searching for the same thing. I at first thought. I can just write a bunch of common aliases in the .bashrc file. Every time someone would run a command like ls I could dump the current history. Then I ran into someone else who was writing the timestamp to the prompt. I figured if you you write a timestamp you may be able to log all history. sure enough I got it.

bash man page
PROMPT_COMMAND
If set, the value is executed as a command prior to issuing
each primary prompt.

# .bashrc

# User specific aliases and functions

alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'

# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi

PROMPT_COMMAND='history 1 >> /${HOME}/hist'

Also, you want to run the same command on the .bash_logout. This way you will not miss the last command.

Edward Capriolo

[ Parent | Reply to this comment ]

Posted by JulienV (83.194.xx.xx) on Mon 4 Jul 2005 at 16:25
[ Send Message | View Weblogs ]
I am used to using `script' for that purpose (only when needed).
It is part of the package bsdutils.
I am pretty sure it is possible to launch it at login time to save typed in commands in a separate file (see -f and -a options).

The only problem I have is when using interactive programs that manipulate the screen (vi, mutt etc.): that creates garbage in the file.

[ Parent | Reply to this comment ]

Posted by gonad (203.118.xx.xx) on Tue 5 Jul 2005 at 06:24
[ Send Message ]
I do the opposite :)

cd ~
rm .bash_history
ln -s /dev/null .bash_history

Sorry this is a sort of parallel topic, but does anyone know how to achieve this in a better way?

[ Parent | Reply to this comment ]

Posted by yarikoptic (4.64.xx.xx) on Tue 5 Jul 2005 at 07:46
[ Send Message ]
doesn't
export HISTFILE=/dev/null
or
export HISTSIZE=0
help?

[ Parent | Reply to this comment ]

Posted by Anonymous (12.206.xx.xx) on Tue 4 Mar 2008 at 19:51
make .bash_history a dir instead of a file to avoid receiving the latest commands. do:

rm ~/.bash_history
mkdir ~/.bash_history

clean and simple.

[ Parent | Reply to this comment ]

Posted by Anonymous (80.242.xx.xx) on Tue 5 Jul 2005 at 15:14
Use bsd-proc-accounting, the kernel must be made with an special option.

You must not hammering on the bash or on ohter shells,
and you be informated if the user foo let's fly a fart,
or an user hammers on the root-account-password to become root-access. All the operations of and on your system get's logged, step by step, so that you can use sar /lastcomm to take the values and digging in the history...:-D

[ Parent | Reply to this comment ]

Posted by xrat (128.130.xx.xx) on Thu 7 Jul 2005 at 17:48
[ Send Message ]
Hi yarikoptic, congratulations! This is a fine article. Your way of saving the history works by far better than just a diff of .bash_history and .bash_history.old.

While trying a different approach, I found that basicly the line history -a $HISTFILE.archive in archive_history() suffices since history -a seems to append only what has not yet been "appended". So, repeated calls of history -a are not producing duplicates.

Furthermore, I am writing markers to the history itself with e.g.

echo "# $USER@${HOSTNAME}:`tty` `date`" >$HISTFILE.tmp
history -r $HISTFILE.tmp ; rm $HISTFILE.tmp
(Is there a way to store a line in the history without creating a temporary file?)

In .bash_logout, I chose to call function archive_history.

Cheerio, and thanks again for your contribution. -- Andreas

[ Parent | Reply to this comment ]

Posted by yarikoptic (4.64.xx.xx) on Thu 7 Jul 2005 at 20:40
[ Send Message ]
Andreas, I knew there must be a cleaner way! Thank you a lot - I'm off to get rid of the monster in favor of your 2 liner

As for tmp file - just out of my head -- you can use bash:
history -r <(echo "# $USER@${HOSTNAME}:`tty` `date`")


but the problem is that I don't see effect of -r at all in the running shell. ie when I -r - nothing changes not in output of history either in .bash_history if I call history -a... I need to try more carefully after I come from the beach :-)

[ Parent | Reply to this comment ]

Posted by yarikoptic (4.64.xx.xx) on Fri 8 Jul 2005 at 09:33
[ Send Message ]
He heh -- is it just my bad luck? :-/
1. history -r doesn't work in .bash* scripts for the same reason: history is not initialized at that moment and then just gets overriden
2. history -r screwes "history -a" call, so it records 1 command less than necessary (if file on -r contained 1 line only, didn't try more...)

3. if you call history -a .bash_history.archive, then it doesn't append to a "working" history file .bash_history, so you have to use something like "history -w" which would be evil because it disregards entries written by other currently running bash sessions.
To overcome this problem: history -a to separate temp file and then append that file to both working and archive histories

4. history -r <( echo line ) doesn't work :-/ it seems internally bash can't use its own creation - this redirected pipes... weird

To overcome mentioned problems and agreeing on history -a to temp file, I've got a nice fully functional (I hope) version (this is a functional part of .bashrc_history script):
if [ "$PS1" ] ; then # interactive shell
    export STARTTIME=`date`
    shopt -s cmdhist
     #histappend
    # Next function really stores the history logs. Besides that if you want to store whatever you have so far
    # you can call this function and it will save increment from the last call to it.
    archive_history()
    {
        TFILE=${HISTFILE}.temp
        CURTIME=`date`
        CURTTY=`tty`
        #STAMP
        echo "#$USER@${HOSTNAME} [ ${STARTTIME} - ${CURTIME} ]:$HISTORYDUMP ($CURTTY) ----" >| $TFILE
        history -a ${TFILE}
        cat ${TFILE} >> ${HISTFILE}.archive
        cat ${HISTFILE} ${TFILE} | tail -${HISTSIZE} >| ${HISTFILE}
        rm -rf ${TFILE}
    }

    trap 'archive_history' EXIT
fi

[ Parent | Reply to this comment ]

Posted by xrat (128.130.xx.xx) on Fri 8 Jul 2005 at 09:49
[ Send Message ]
I am running GNU bash 2.05b.0.

history -r <(echo "# $USER@${HOSTNAME}:`tty` `date`")
seems to have no effect, whereas
echo "# $USER@${HOSTNAME}:`tty` `date`" >tmpfile
history -r tmpfile ; rm tmpfile
actually puts the message into the history. ^p (C-p) then shows this line. Looks like history -r needs a file. Cheerio, -- Andreas

[ Parent | Reply to this comment ]

Posted by yarikoptic (4.64.xx.xx) on Fri 8 Jul 2005 at 10:08
[ Send Message ]
yeap - -r needs a file and <() creates a pipe (ie a file which gets its data from output of a command). For most cases (when that file is read just once) that is just as fine but for this one - nope (as I said in the previous post)

[ Parent | Reply to this comment ]

Posted by xrat (62.178.xx.xx) on Fri 17 Aug 2007 at 21:49
[ Send Message ]
As of 2007-08-15, there is a new article (and discussion) with new approaches: Bash eternal history

-- Andreas

[ Parent | Reply to this comment ]

Posted by Anonymous (89.247.xx.xx) on Sat 12 Jan 2008 at 10:42
Hi.

I once had a similar setup. To have my shell history even survive the livetime of the particular machine captured from, I ended up having a line like "mail -s "'actions on `hostname` on `date`" acct@domain.tld < ~/.bash_history.asc && rm ~/.bash_history" in my .logout.

This setup was running in a potentially hostile shared environment,so the history file was ecncrypted with pgp before transport, for it might contain things like host/login combinations etc. which you might not like to be readable with a plain vanilla tcpdump.

This was 10 years ago, now I more use like accounting.

[ Parent | Reply to this comment ]

Posted by Anonymous (66.37.xx.xx) on Tue 1 Apr 2008 at 18:05
In working with this, my variable $HISTCMD is always set to 1. So, when I exit, I get the following (I've added a couple echo's to determine why I'm getting the problem). If I type 'echo $HISTCMD' at the command line, I get the appropriate number.

HISTCMD 1
-499
bash: history: -4: invalid option
history: usage: history [-c] [-d offset] [n] or history -awrn [filename] or history -ps arg [arg...]

Any ideas?

Thanks,
Dave

[ Parent | Reply to this comment ]

Posted by giosue_c (68.255.xx.xx) on Mon 5 Jan 2009 at 05:51
[ Send Message ]
I think this project is trying to achieve what you are describing: http://shell-sink.blogspot.com/ It is a tool that keeps your bash history in a searchable database out on the web. Check it out.

[ Parent | Reply to this comment ]

Posted by Anonymous (208.105.xx.xx) on Mon 14 Dec 2009 at 20:54
you should check psacct/acct to log all process/cpu time/user/timestamp ... kky

[ Parent | Reply to this comment ]

Posted by Anonymous (194.42.xx.xx) on Thu 5 Aug 2010 at 09:13
Dear yarikoptic,

I get an error when using this with BASH on Ubuntu 10:
bash: /home/benyg/.bashrc: line 175: unexpected EOF while looking for matching `)'
bash: /home/benyg/.bashrc: line 183: syntax error: unexpected end of file

line 175 = the CURBASHDATE line.

Do you know why it might not work on this version of BASH?

Thanks,

Benjamin Goodacre

[ Parent | Reply to this comment ]

Posted by BenyG (194.42.xx.xx) on Mon 16 Aug 2010 at 09:52
[ Send Message | View Weblogs ]
The version that you have here worked for me instead: http://www.onerussian.com/Linux/bash_history.phtml

Thank you,

Benjamin Goodacre
http://ben.goodacre.name/tech

[ Parent | Reply to this comment ]

Posted by yarikoptic (129.170.xx.xx) on Mon 16 Aug 2010 at 15:54
[ Send Message ]
For the latest revision of the script please look at git repository. I have added a link to it from the elderly page on my website, i.e. http://www.onerussian.com/Linux/bash_history.phtml

[ Parent | Reply to this comment ]

 

 

Flattr