The software running the decoding process (mpg123) can happily work on a 486DX/66 or higher without any constraints. 16MB of RAM ought to do fine as well. Any sound card that is supported under Linux should work. The following list includes all the hardware I used:
Pentium 120 downclocked to 90 Mhz (I needed to lower the temperature of the CPU so that I wouldn't have to use a fan - just a heatsink). Cost: $5.
Unknown brand-name motherboard (AT-style, 4 PCI, 3 ISA, 4x DIMM, Award BIOS) - if you are going to get rid of the video card (after you have configured the machine) make sure you change the display from CGA/EGA (or VGA) to None. Cost: $10.
Small powersupply (50 or less watts would do). I got one from a Compaq computer (the wiring on the powersupply didn't match the AT P8/P9 connector standards, thus I had to match the cables and rewire them to fit my AT motherboard). You could probably use any powersupply as long as it gives you 12VDC and ground. You won't need the 5VDC unless you have a keyboard, mouse, or floppy connected. Cost: FREE (found the computer in a dumpster).
Intel EtherExpress Pro 10/100 Management Network Card. Any network card will do as long as it has PXE or BOOTP functionality - those things are also known as network booting. Without this kind of network functionality, you would need to attach a harddrive/zip-drive/floppy-drive to boot the operating system from. Cost: $15 (Intel had a promotion and they were selling two for $30).
Sound card. I got a Crystal CS424x card b/c its height was pretty small compared to other sound cards. Cost: $15.
Remote control. This is the difficult choice - if you pick a Packard Bell remote, you will end up pulling your hair out. I got a Logitech AST Remote. Before you get a remote, you ought to check for software that will work with your remote. Cost: $5.
Cabling - stereo RCA cables from RadioShack. Network cable and a jack from local distributor of electronic components. Cost: $9.
Small 8mm screws to screw the custom box together. Picked up at Home Depot along with plexiglass. Cost: $2.
Plexiglass - used to construct the custom box. Bending of the box is pretty easy, as long you have a torch and don't burn your own eyelashes with it. Picked it up at Home Depot. Cost: $10.
Black paint so the box would look inconspicuous and color-match my stereo rack. Cost: $2.