Size of the computer component (I was tempted to use PC104 boards, but they were to expensive). Height of ISA/PCI cards (so that the box doesn't look like a baby-AT case).
The appealing side of the box - does it look like a square rectangle, pyramid, triangle, or something even more weird?
Paint. Do you want see-through box with flashing lights inside, inconspicuous black box, or chrome looking?
Cost. The absolute minimum (taking into consideration that you, the reader, has no spare computer parts) is about $70 (excluding tax, shipping, etc)
KISS (Keep It Simple and Stupid) - take everything that's unnecessary and get rid of it (parallel port, modem card, video card, etc).
The idea behind this box is to separate tasks - the MP3-box can only play the mp3's from a server, while you, the reader, can freely add more files to your big-end server. Thus, the MPEG-box wouldn't need a harddrive, nor floppy drive. It would boot the operating system from the network (from the same server where the mp3 files are located).