This is the most variable part of the job, and all I can really do to help is to describe what I did in my system, in the hope that it will serve as a rough guide. Basically, you want to look at every directory that contains any
files that aren't part of your standard linux installation, or
files that are actually newer than the ones you'll install when you do your new linux installation.
and separate out only those files that you want to carry over.
(Another possible strategy is to back up all files with mtime or ctime more recent than the day of the previous linux installation, as mentioned above, and then restore from that. If you do that, you have to take into account that the new linux distribution may contain versions of some files that are newer still than the ones you saved.)
In my case, I ended up making a .tgz file on the backup medium for each of
/usr/lib/trn (the rest of /usr/lib would be reinstalled)
/usr/local/thot (there were other /usr/local files I didn't need)
/usr/src/lilo-17 (because my new Slackware still had version 16)
/usr/src/linux-1.2.13 (because I'd done some customizing)
/usr/X11R6/lib/X11/initrc (the rest of Xfree86 was to be reinstalled)
My machine was relatively easy in that there were no spool files to worry about. I don't run a news spool on this box, and since there are only two users, it was easiest just to get all the mail read before shutting down. Otherwise, /var/spool directories would have had to be backed up at the last minute. (And, of course, the news library and site directories!)