After setup the DNS account from hn.org, I change the DNS entry, both of the primary and secondary server to the DNS server provided by hn.org from the domain registration company (usually register.com or whatever). It may take some times to get the DNS entry refresh.

Wonderful! Now the DNS entry was refreshed and all request to www.myfakedomain.com will forward to my baston host. That's simple huh? Thanks for the great work of hn.org. For details about how to setup DNS entries, please refer to DNS-HOWTO.

Because the machine connected to ADSL modem provide services for the public, that mean it will be accessed by anyone who have Internet access from anywhere. I need to restrict the access of various tcpd services for this machine for security reason. I edited the file /etc/hosts.allow and /etc/hosts.deny accordingly:

/etc/hosts.allow

ALL: 127.0.0.1
in.telnetd: 192.168.2.2
in.ftpd: 192.168.2.2
sshd: 192.168.2.2 203.xxx.xxx.xxx

/etc/hosts.deny

ALL: ALL : spawn (echo Attempt from %h %a to %d at `date` | tee -a /xxx/xxx/tcp.deny.log | mail my@email.com )

As shown from the above configuration files, all machines from internal network can telnet, ftp, ssh and sftp to the baston host. The address 203.xxx.xxx.xxx is the I.P. address of my office machine which is allowed to remote login to the baston host using ssh and transfer file to the baston host using sftp. Telnet and ftp to the baston host will never allow from machine outside the internal network because user name and password is transmit in plaintext format. It may be captured by hacker easily. HTTPD is not included in the above configuration file because HTTPD is not under controlled of INETD.

Connect to the baston host safely using SSH

Telnet and FTP is allowed to connect to the baston host from the internal network. SSH and SFTP must be used to connect from external network. Refer to the article 'Using ssh' from Linux Gazette about how to setup and usage of SSH. You must install and running SSHD in order to support SSH. SFTP can be download from http://enigma.xbill.org/sftp/. SFTP is easy to use and install, please refer to the readme from the web site.

Setup the Intranet Server

In order to protect the internal network, I disable all access from external network to my internal network:

/etc/hosts.allow

ALL: LOCAL 192.168.1.2 192.168.1.7

/etc/hosts.deny

ALL: ALL : spawn (echo Attempt from %h %a to %d at `date` | tee -a /xxx/xxx/tcp.deny.log | mail my@email.com )

An email will be sent to my mailbox in case there are any activities attempt to connect to any prohibited services to both of my linux server.

As shown from figure 1, all internal machines have a host name. You can use whatever host name and domain name for your internal network even the domain name is already registered at NIC, however, special care must be taken when setting up your own internal DNS server.

Setting up intranet DNS server - named

Again, please refer to the HOWTO or technical books about how to setup a DNS server. Following shows my configuration files of the DNS server running at the Intranet server:

/etc/named.boot
/etc/named.conf
/var/named/named.ca
/var/named/named.local
/var/named/named.myhome.net
/var/named/named.rev.1
/var/named/named.rev.2

More security issues

Hackers are arounding you, only firewalling with packet filtering and controlling services access from hosts.allow/hosts.deny are never enough. A few security holes may discover everyday. You should subscribes to corresponding mailing list and upgrade your linux constantly. A few more articles and software about security is good and worth to introduce:

• Security for the Home Network LG #46
  
• Linux Firewall and Security Site
  
• Mason - the automated firewall builder for Linux
  
• Astaro AG (Great firewall linux distribution with web interface)
  
• The Ethereal Network Analyzer
  
• Nessus - The Security Scanner
  
• Stunnel - Universal SSL Wrapper
  

  How about POP3 and SMTP server?

  POP3, as same as TELNET and FTP, transfer username and password in plaintext and is considered insecure. SPOP maybe setup to encrypt POP data. However, I don't want to store my personal email in any machine outside internal network including my office's workstation. So I'm not going to setup POP3 in the baston host. The reason not to allow SMTP because relaying mail is dangerous because spammer will make use of your relayed SMTP server to send their hateful spam mails. On the other hands, setting up a non-relayed SMTP server for yourself is meaningless because you cannot send mail from your SMTP server outside the network. I can simply login to my baston host using ssh and run pine to check and reply my message in a secure way.

  Subdomain for web server

  Wow, everything working now. I can host my web server, email server and ftp server at my home linux box. It rocks! Now I need a subdomain resume.myfakedomain.com to host my online resume. Just add the following lines to the /etc/httpd/conf/httpd.conf handles all the magic:

  RewriteEngine on
  ## Ignore www.myfakedomain.com
  RewriteCond %{HTTP_HOST} !^www\.myfakedomain\.com [NC]
  ## A directory with the name of the subdomain must exist
  RewriteCond %{DOCUMENT_ROOT}/%1 -d
  ## Add the requested hostname to the URI
  ## [C] means that the next Rewrite Rules uses this
  RewriteRule ^(.+) %{HTTP_HOST}/$1 [C]
  ## Translate abc.myfakedomain.com/foo to myfakedomain.com/abc/foo
  RewriteRule ^([a-z-]+)\.myfakedomain\.com/?(.*)$ http://www.myfakedomain.com/$1/$2 [L]
  

  Other useful configuration files

  /etc/hosts (baston host)

  127.0.0.1       localhost.localdomain   localhost
  192.168.2.1     router.myhome.net       router
  192.168.2.2     gateway.myhome.net      gateway
  202.xxx.xxx.xxx www.myfakedomain.com    www
  

  /etc/hosts (intranet gateway)

  127.0.0.1       localhost.localdomain   localhost
  192.168.1.1     server.myhome.net       server
  192.168.1.2     devel.myhome.net        devel
  192.168.1.3     php.myhome.net  php
  192.168.1.4     asp.myhome.net  asp
  192.168.1.7     be.myhome.net   be
  192.168.2.1     router.myhome.net       router
  192.168.2.2     gateway.myhome.net      gateway
  

  /etc/resolv.conf (baston host)

  search myfakedomain.com
  nameserver      127.0.0.1
  

  /etc/resolv.conf (intranet gateway)

  search  myhome.net
  nameserver      127.0.0.1
  

  Network Card Setting

  Ethernet port setting:

  

  More network configuration files:

  /etc/sysconfig/network (baston host)
  /etc/sysconfig/network-scripts/ifcfg-eth0 (baston host)
  /etc/sysconfig/network-scripts/ifcfg-eth1 (baston host)

  /etc/sysconfig/network (Intranet gateway)
  /etc/sysconfig/network-scripts/ifcfg-eth0 (Intranet gateway)
  /etc/sysconfig/network-scripts/ifcfg-eth1 (Intranet gateway)

  /etc/rc.d/rc.local (Both of the Baston host and Intranet gateway)
  

  TCP/IP setting summary

  Baston host
  Default Gateway:	ppp0
  Nameserver:	127.0.0.1
  Network interface:	eth0
  I.P. Address:	192.168.3.1
  Subnet mask:	255.255.255.0
  Network interface:	eth1
  I.P. Address:	192.168.2.1
  Subnet mask:	255.255.255.0

  Intranet Server
  Default Gateway:	192.168.2.1
  Nameserver:	127.0.0.1
  Network interface:	eth0
  I.P. Address:	192.168.1.1
  Subnet mask:	255.255.255.0
  Network interface:	eth1
  I.P. Address:	192.168.2.2
  Subnet mask:	255.255.255.0

  Workstations from Internal Network
  Default Gateway:	192.168.1.1
  Nameserver:	192.168.1.1
  Network interface:	eth0
  I.P. Address:	192.168.1.X
  Subnet mask:	255.255.255.0

  Further setup and reading

  What if you want to access your internal machine running windowsz from the other network while maintaining security through the firewall? The answer is using Virtual Private Network (VPN) technology. Linux do support VPN in recent version. More details can be find at VPN HOWTO. If you have more than one domains and want to host at the same baston host, you may require special setting for your apache web server and sendmail server. The next version of this article will include the walkthrough of the VPN and virtual domain setup.

  If you have any suggestions or comments regarding this document, please feel free to contact me at .

  ---

  Copyright © 2001, Ray Chan.
  Copying license http://www.linuxgazette.com/copying.html
  Published in Issue 65 of Linux Gazette, April 2001

  "Linux Gazette...making Linux just a little more fun!"

  ---

  HelpDex

  By

  ---

  The first cartoon is a reference to the Cobalt Qube, a network server appliance that runs on Linux.
  
  
  
  

  Shane's cartoon archive is available on his web site, http://www.ShaneCollinge.com/.

  ---

  Copyright © 2001, Shane Collinge.
  Copying license http://www.linuxgazette.com/copying.html
  Published in Issue 65 of Linux Gazette, April 2001

  "Linux Gazette...making Linux just a little more fun!"

  ---

  Interview with Linux Today's Marty Pitts

  By and Marcos Martins Manhães
  Originally published at OLinux

  ---

  Enjoy this interview with Marty Pitts, Managing Editor at Linux Today. He talks about Linux Today's evolution and the growth of its main subject, Linux operating systems.

  OLinux: Please introduce yourself. (career, education, hobbies, personal and professional achievements).

  Marty Pitts: My name is Marty Pitts.I worked in the nuclear industry for 13 years before joining Linux Today, in jobs ranging from Purchasing Agent to Network Admin. I like to ski in the winter, hike and camp in the summer and read SciFi in between.I also like to play around with the latest Linux distros.

  OLinux: How long have you been working and what are your responsibilities at Linux Today?

  Marty Pitts: When I became interested in using Linux at work, I started looking for a information about Linux online.One of the resources I came across was Linux Today. I liked that it was updated hourly. When I found news that they did not have, I started using their contrib form.After several months, the site owners: Dave and Dwight, asked if I would be interested in working as a volunteer on the site. Having become a Linux news junkie, I jumped at the chance.

  In the summer of 1999, Dave sent me an email asking what my employment situation was.It just so happened that at my current job, my boss of several year had just turned in his notice to quit.It was a good opportunity to think about a career change.How many people actually get a chance to work at what they love?

  I started working for Dave and Dwight in September of 1999 full time as the Managing Editor.About a month later, Dave and Dwight sold the Linux Today web properties, which included LinuxPR.com, to internet.com. I have been as a full time employee of internet.com ever since.

  OLinux: How's the site organized? Give us an idea of how the Linux Today works. How many people are involved?

  Marty Pitts: For the whole channel, which includes 14 web sites, there are approximately 9 full time editors and programmers.

  Right now there are two full time people who work on Linux Today, myself and Michael Hall.We also take care of LinuxPR and a couple of other sites in the Linux/Open Source channel.

  Michael lives on the east coast of the US, and I live in Washington state on the west coast.So naturally we break up the day, with Michael covering the first part of the day and then I come online later with a couple hours of overlap.

  OLinux: Can you describe Linux Today evolution since it began?

  Marty Pitts: Dave and Dwight were the ones that came up with the idea for Linux Today and they are the ones who successfully executed that idea.They were successful enough to attract the attention of internet.com.

  It started as a labor of love for Dave and Dwight.They wanted to provide a resource that people could use to find out what was going in the Linux/Open Source world.They started the site on September 30, 1998. A year later, they had both quit their daytime jobs to work full time on the site, they had been able to hire a full time editor, and they had posted over 10,000 stories. Currently we are right at 34,000 stories posted, just on Linux Today.

  After the sale of the site to internet.com, somethings changed and others remained surprisingly the same.Dave chose to leave and pursue other goals, Dwight stayed on and we worked to keep the site going. To replace Dave, who had done most of the site programming, Paul Ferris was hired.

  Paul, a great guy, started working on the programming side but still found time to write his column: Rant Mode Equals One.Currently we are using the second iteration of the site code, which Paul wrote, and we are about to roll out the third iteration of the code.It will provide increased flexibility so that the code will be able to be used across a variety of different sites, each with its own unique requirements.

  What stayed the same, during the transition, was the direction and focus of Linux Today. We were told to keep doing what we had been doing that had made Linux Today a popular site, which was a relief.

  Today we have a lot more original content than we used to. In addition, our focus is on making the whole Linux/Open Source channel work together well.

  OLinux: Are there companies sponsoring or maintaining Linux Today?

  Marty Pitts: Since Linux Today is owned by internet.com, they are the ones who pay for the maintenance of this and the other sites.

  OLinux: Is there any central control to avoid redundancy and improve editorial efforts?

  Marty Pitts: Yes. We have, as part of the backend to Linux Today, an Editorial Board that keeps track of who is working on what stories. In addition, we use email extensively, plus we have an IRC channel for quick communication.

  In spite of that, we occasionally will have a duplicate story go up.Which is why, sometimes you will see a message that says, 'This story has been unposted.'

  OLinux: How difficult is to present good content day by day? Besides the users' contribution, do you have any other content resource (agencies like Reuters, etc.)?

  Marty Pitts: Early in the week, Monday and Tuesday, it is usually very easy to find content to post.As the week progresses though, it can be a struggle to find good content and resist posting something that is just a rehash of a story that has already been covered.Weekends are more difficult since there is usually no news from the traditional sources. Since we like to have time off as well, we break up the weekend between the editors and we also future post some items so that they show up over a regular period of time.This way, we are able to take a break and our readers can find some fresh content.

  Our readers are a very valuable source of content.Without them providing links and suggestions, Linux Today would not be were it is today.

  We are able to find some relevant content elsewhere within the internet.com properties, which we use when available.

  OLinux: How do you see Linux Today in the Open Source world? What's the best contribution Linux Today has been giving to the Linux community during its existence?

  Marty Pitts: We see Linux Today as the place to stop if you want to know what is going on in the Linux/Open Source community today. We search out the events and news and bring it to one place so that our readers don't have to spend the time doing that search for themselves.

  Through the forums and story talkbacks, we help to facilitate discussions within the community and give our readers a place to react to the news of the day.

  I believe that Linux Today's greatest contribution is that we are able to raise the awareness of our readers about the events, good and bad, that are happening within and to the Linux/Open Source community.

  OLinux: What are the new features being developed for Linux Today? Can you detail the main currently projects?

  Marty Pitts: We are about to roll out redesigned site software that will provide a greater flexibility and robustness to all the sites on the channel.

  OLinux: What is your opinion about the growth of Linux in the enterprises? What about desktops, do you have a projection for the future?

  Marty Pitts: From my experience, Linux is infused into the enterprise deeper than anyone suspects. When a problem can be solved without having to ask for a new budget item, the guys/gals on the front lines will use what works. I see the projections by companies like Gartner and IDC and I have to laugh. They don't know how to properly measure the revolution that is taking place under their noses. Their methodology can't account for stealth deployments.

  The desktop is there already. Ease of use and graphical tools have come a long way in just the past year or two. I use Linux as my work environment, and for many like myself, Linux is already there.Just look at what we have available to us, DVD decoding and playback capability, the latest video, sound and networking hardware. The environments available are amazing as well. Even though I don't use KDE or GNOME (I use a pure 'Enlightenment' desktop), I have both of them on my system and use their apps.

  ---

  Copyright © 2001, Fernando Ribeiro Correa.
  Copying license http://www.linuxgazette.com/copying.html
  Published in Issue 65 of Linux Gazette, April 2001

  "Linux Gazette...making Linux just a little more fun!"

  ---

  Internet Printing - Another Way

  By

  ---

  The Problem

  You are doing some work on your home PC, connected to your favorite ISP - and you decide you want to print a Word document on the high-speed color printer at your office. That printer is connected to the corporate LAN, but you can't talk to it using LPR or IPP because it is hidden behind the corporate firewall.

  You could perform a print-to-file operation, then email the file to somebody at your office, and get them to send it to the printer. But there are a few steps here - and it gets more complicated if there is a restriction on the length of email messages which can be passed through one of the servers along the way. You will then have to perform some sort of file-split operation and send the individual parts.

  Client Software

  The people who make Brother printers thought of all this, and developed a set of Windows printer drivers. These enable users to print directly to a designated email address. The print-job is automatically split into parts if necessary, and each part is base64-encoded prior to transmission. Users can also nominate an address for email confirmation.

  These Windows printer drivers (for Windows 95/98, and for Windows NT-4.0/2000) can be downloaded from the Brother website.

  Printer Capabilities

  What the Brother people expect users to do their printing on is, of course, a Brother printer - specifically, in this instance, one equipped with a network card able to accept, decode and re-assemble mail messages directed to it.

  But what if you wish to print on a printer from another manufacturer?

  Doing it in Software

  My first stab at this was a Korn-shell program to which appropriate incoming mail items were piped via a sendmail alias. The program used 'awk' to extract information such as job and part number, then decoded each such item into an appropriately named file in a designated directory.

  After receiving a part, the program marked it as "complete", then set an anti-simultaneity lock and went through a procedure to determine if all necessary parts had been received in full. If they had, it concatenated them in sequence, piped the result to the nominated printer, and deleted them.

  It was then that I started thinking: "What if there isn't enough room to store all the parts for all the jobs which may currently be arriving?" And: "How do the Brother people do it on a network card?"

  Doing it Without Local Storage

  The answer to my second question is: "They use a POP3 server!". The components of each job stay on that server until the network card determines that all necessary parts are available, at which stage it sucks them down and decodes them in sequence, sending the output to the printer mechanism, and requests their deletion from the server.

  So here's how it can be done on a Linux machine. The program has been written in Perl so that the NET::POP3 module can be used for easy access to a POP3 server. It has been tested on both NetBSD and Solaris machines, so it should work almost anywhere; all you'll have to change are the location of the Perl interpreter, the name used for 'awk', and the format of the 'lpr' command. [Text version of this listing.]

  #!/usr/bin/perl -w
  # @(#) BIPprint.pl      Acquires Brother-Internet-Print files from POP3 server
  #                       and passes them to designated printer(s). Small-memory
  #                       version.  Intended for invocation via inittab entry.
  #                       Graham Jenkins, IBM GSA, Feb. 2001. Rev'd: 17 Mar. 2001.
  
  use strict;
  use File::Basename;
  use Net::POP3;
  use Date::Manip;
  use IO::File;
  my $host="bronzeback.in.telstra.com.au";        # Same host and password for
  my $pass="MySecret";                            # each printer.
  my $limit=30*1024*1024;                         # Maximum bytes per print job.
  my ($printer,$awkprog);
  defined($ARGV[0]) || die "Usage: ", basename($0). " printer1 [ printer2 ..]\n";
  open(LOG,"|/usr/bin/logger -p local7.info -t ".basename($0)); autoflush LOG 1;
  $awkprog="";                            
  while (<DATA>) {$awkprog = $awkprog . $_};      # Build awk program for later,
  while (1) {                                     # then loop forever, processing 
    sleep 30;                                     # all printers in each pass, and
    foreach $printer (@ARGV) {process($printer);} # sleeping for 30 seconds
  }                                               # between each pass.
  
  sub process {
    my ($flag,$i,$j,$k,$l,$m,$allparts,$user,$pop,@field,@part,$count,$top15,
        $msgdate,$parsdate,$notify,$reply,%slot,$fh);
    $user = $_[0];
    $pop = Net::POP3->new($host);                 # Login to POP3 server and get
    $count = $pop->login($user,$pass) ;           # header plus 1st 15 lines
    $count = -1 if ! defined ($count) ;           # of each message. Use apop
    for ($i = 1; $i <= $count; $i++ ) {           # instead of login if server
      $top15=$pop->top($i,15) ;                   # supports it.
      if ($top15) {                       
        $msgdate = ""; $notify="None"; $reply="";
        for ($j = 0; $j < 99; $j++ ) {
          if (@$top15[$j]) {                      # Use arrival-date on POP3
            if($msgdate eq "") {                  # server to ascertain age of
              (@field) = split(/;/,@$top15[$j]);  # message; if it is stale,
              if ( defined($field[1])) {          # delete it and loop for next.
                $parsdate=&ParseDate($field[1]);  # (Search for semi-colon
                if( $parsdate ) {                 # followed by valid date.)
                  $msgdate="Y";
                  if(&Date_Cmp($parsdate, &DateCalc("today","-3 days") ) lt 0 ) {
                    print LOG "Stale msg: $user $parsdate\n";
                    $pop->delete($i);
                    goto I;                       # If POP3 server does
                  }                               # automatic message expiration
                }                                 # this entire section can be
              }                                   # omitted.
            }
            (@field) = split(/=/, @$top15[$j]);
            if ( defined($field[0]) ) {   
              if ($field[0] eq "BRO-NOTIFY") {chomp $field[1];$notify=$field[1];}
              if ($field[0] eq "BRO-REPLY")  {chomp $field[1];$reply =$field[1];}
              if ( $field[0] eq "BRO-PARTIAL" ) { # Comment above line to
                ( @part )=split("/", $field[1]);  # prevent mail notification.
                chomp $part[1];           
              }
              if ( $field[0] eq "BRO-UID" ) {     # Determine print-job and part
                chomp $field[1];                  # thereof contained in message.
                $slot{$field[1]."=".$part[0]} = $i ;
                $allparts = "Y";                  # As we see each message, check
                for ($k=1;$k<=$part[1];$k++) {    # whether we have all parts.
                  $allparts = "N" if ! defined($slot{$field[1]."=".$k}) ; 
                }
                if ( $allparts eq "Y" ) {         # Print and delete all parts.
                  print LOG "$field[1] $part[1] => $user\n";
                  if(($notify ne "None") && ($reply ne "")) {system 
                    "echo Print Job Received, $part[1] pcs|Mail -s$user $reply";}
                  $fh=new IO::File "|awk '{$awkprog}' Limit=$limit |lpr -P $user";
                  for ($k = 1;$k<=$part[1];$k++) {
                    $pop->get($slot{$field[1]."=".$k},$fh) ;
                    $pop->delete($slot{$field[1]."=".$k}) ;
                  }                               # If there is enough filespace,
                  $fh->close;                     # pipe awk output thru gzip to
                }                                 # a temporary file, then print
              }                                   # it and delete all parts; this
            }                                     # caters for connection failure.
          }                       
        }                                         # The awk program here-under
      }                                           # is used to extract parts from
  I:}                                             # a file containing multiple
    $pop->quit() if ($count >= 0);                # parts and feed each of them
  }                                               # through a decoder to stdout.
  __DATA__
  if( Flag == 2 ) {
      Size=Size+length
      if(length == 0) { Flag=0; close("mmencode -u 2>/dev/null") }
      else if(Size<=Limit*4/3) print $0 |"mmencode -u 2>/dev/null" }
    if( Flag == 1 ) if(length == 0) Flag=2
    if( Flag == 0 ) if($1 ~ /^Content-Transfer-Enc/) if($NF == "base64") Flag=1
  

  Program Walk-Through

  The program builds a small 'awk' program for later use; then, for each printer declared on it's command line, it accesses a mailbox of the same name and examines each message therein. If a message is stale, it is deleted. Otherwise the contents of some Brother-specific lines are extracted; these indicate whether email notification is required, and which part of which job is contained in the message.

  If, during examination of a message, it is determined that all the parts of its corresponding job have been seen in the mailbox, an email notification is generated if required, and the parts are extracted in sequence and piped via the 'awk' program (which decodes each part as it arrives) to an appropriate printer command. Each part is deleted as soon as it has been processed in this manner.

  Ideally, we should wait until success (or other) notification of print submission was obtained before performing the email and deletion tasks; however, as noted in the listing, this requires some local storage. In a like vein, whilst the Brother client software allows selection of email notification for several different conditions, we send notification of job submission unless "None" has been selected.

  Concluding Remarks

  This program contains a password, so it should be readable only by the user who will execute it. No special privileges are required for execution, and your entry for it in /etc/inittab should look something like:

  bi:345:respawn:su - nobody -c "/usr/local/bin/BIPprint.pl lp1 lp2 >/dev/null 2>&1"

  If you have read this far, you are probably saying: "OK, so the program doesn't need much local storage - but it sends its output to a print spooler! How bad is that?" If the size of your spool area is of concern, you can use something like 'netcat' or 'hpnpout' to send the job directly to a printer port instead of spooling it. Or you may be able to pipe your job through an FTP connection to your printer. If you do bypass the spooler in this fashion, you should use a separate instance of the program for each printer.

  It's not rocket science, and there's no user-authentication or content-encryption. But it may make your life a little easier. Enjoy!

  ---

  Copyright © 2001, Graham Jenkins.
  Copying license http://www.linuxgazette.com/copying.html
  Published in Issue 65 of Linux Gazette, April 2001

  "Linux Gazette...making Linux just a little more fun!"

  ---

  Parallel Processing on Linux with PVM and MPI

  By

  ---

  This article aims to provide an introduction to PVM and MPI, two widely used software systems for implementing parallel message passing programs. They enable us to use a group of heterogeneous UNIX/LINUX computers connected by a network as a single machine for solving a large problem.

  1. Introduction to Parallel Processing

  Parallel processing is a form of computing in which a number of activities are carried out concurrently so that the effective time required to solve the problem is reduced. In the previous days, parallel processing was used for such thing as large scale simulations (e.g. molecular simulations, simulation of the explosion of an atomic bomb etc), solving large number crunching and data processing problems (e.g. compiling the census data) etc. However, as the cost of hardware is decreasing rapidly, parallel processing is being uses more and more in routine tasks. Multiple processor servers have been in existence for a long time. Parallel processing is also used in your own PC too. For example, a graphics processor working along with the main processor to render graphics on your monitor is also a form of parallel processing.

  However, apart from the hardware facilities for parallel processing, some software support too is required so that we can run the programs in parallel and coordinate their execution. Such a coordination is necessary due to the dependencies of the parallel programs on one other. This will become clearer when we work through an example. The most widely used method to achieve such coordination is message passing in which the programs coordinate their execution and in general communicate with each other by passing message's to one other. So, for example, a program may tell another program, ``Ok! Here is the intermediate result you need to proceed.'' If all this sounds too abstract, lets proceed with a very simple example.

  2. A Very Simple Problem

  In this section, we will consider a very simple problem and consider how we can use parallel processing to speed up its execution. The problem is to find the sum of a list of integers stored in an array. Let us say that there are 100 integers stored in an array say items. Now, how do we parallelize this program? That is, we must first find out a way in which this problem can be solved by a number of programs working concurrently. Many a times, due to data dependencies, parallelization becomes a difficult problem. For example, if you want to evaluate (a + b) * c, which involves two operations, we cannot do them concurrently, the addition must be done before the multiplication. Fortunately, for the problem that we have chosen, parallelization is easy. Suppose that 4 program or processors will be working simultaneously to solve the addition problem. Then the simplest strategy would be to break the array items into 4 parts and have each program process one part. Thus the parallelization of the problem is as follows:

  1. Four programs say P0, P1, P2 and P3 will solve the problem.
  2. P0 will find the sum of array elements items[0] to items[24]. Similarly, P1 will find the sum of items[25] to items[49], P2 items[50] to items[74] and P3 items[75] to items[99].
  3. After these programs have executed, there must be some other program to find the sum of the 4 results obtained and give the final answer. Also, the elements of the array items are not known to the programs P0 to P3 and hence some program must tell these programs the values of the elements. Thus, apart from P0 to P3, we will require one more program that distributes data, collects results and coordinates execution. We call such a program as master and the programs P0 to P3 as slaves and this organization as the master - slave paradigm.

  With this organization in mind, let us write the algorithms for the master and the slave programs.

  ---

  /* Algorithm for the master program */
  initialize the array `items'.
  
  /* send data to the slaves */
  for i = 0 to 3
      Send items[25*i] to items[25*(i+1)-1] to slave Pi
  end for
  
  /* collect the results from the slaves */
  for i = 0 to 3
      Receive the result from slave Pi in result[i]
  end for
  
  /* calculate the final result */
  sum = 0
  for i = 0 to 3
      sum = sum + result[i]
  end for
  
  print sum
  

  ---

  The algorithm for the slave can be written as follows.

  ---

  /* Algorithm for the slave program */
  
  Receive 25 elements from the master in some array say `items'
  
  /* calculate intermediate result */
  sum = 0
  for i = 0 to 24
      sum = sum + items[i]
  end for
  
  send `sum' as the intermediate result to the master
  

  ---

  3. Implementing with PVM

  Now that the basic algorithm has been designed, let us now consider how we can implement it. What hardware shall we run this program on? Clearly, very few of us have access to special machines designed to run parallel programs. However, no special hardware requirements are there in order to implement this program. A single computer or a group of interconnected computers will do, thanks to PVM, a software system that enables us to use interconnected computers for parallel program execution. PVM stands for Parallel Virtual Machine. It enables you to create number of programs or processes that run concurrently on same or different machines and provided functions with which you can pass messages among the processes for coordination. Even if you have a single computer, PVM will work on it, although there will be no ``real'' parallel processing as such. However, for learning purpose, that should be fine. Later on I will describe how to do ``real'' parallel processing using the PVM.

  In order to use the PVM system, you need to install the PVM software on your Linux system. In case you are using Red Hat Linux, then the RPM package for PVM is included on the CD, so that you can install it as you normally install other packages. Assuming that you have installed PVM system on your machine, create the following directories(s) in your home directory: ~/pvm3/bin/LINUX/. Why ? Because PVM requires that some of the executables you create be copied in this directory. Once you have done this, your setup is ready. Test this by giving the command pvm on the prompt. This will start the PVM Console from which you can give commands to the PVM system and query status information. If everything is set OK, you will see the pvm> prompt. Here enter the command conf. The output should look something like this.

  pvm> conf
  conf
  1 host, 1 data format
                      HOST     DTID     ARCH   SPEED       DSIG
                 joshicomp    40000    LINUX    1000 0x00408841
  

  What does this mean? The PVM System allows you to consider a group of interconnected LINUX system to be viewed as a ``virtual'' computer having much higher computing capacity than the individual machines. Thus, PVM will distribute the processes among a number of computers. However, by default, PVM considers that only the host that you are working on is to be included in the PVM machine, i.e. all processes you create will be scheduled to run on the same host. The conf command shows what hosts or nodes are in the PVM. Currently, there is only one. Later on, we will see how to add more hosts. Presently, exit the PVM console by giving the command halt

  3.1 A Demonstration Program

  Now that you are ensured that the PVM system has been properly installed, let us see how to write the programs. Programs for the PVM system can be written in both FORTRAN and C. We will be using the C language. To use the PVM system, you include some calls to the PVM functions in your C program along with the other statements and link the PVM library with your programs. To get you started with PVM, let us write a simple program in which there will be a master and a slave. The master will send the slave some string, which the slave will convert to upper case and send back to the master. The master and the slave programs are given as follows. To compile the programs, give the command make -f makefile.demo.

  [Click here for a tar file containing the program listings.]

  ---

        1 /* -------------------------------------------------------------------- *
        2  * master_pvm.c                                                         *
        3  *                                                                      *
        4  * This is the master program for the simple PVM demonstration.         *
        5  * -------------------------------------------------------------------- */
        6 #include <stdio.h>
        7 #include <stdlib.h>
        8 #include <pvm3.h>           /* declares PVM constants and functions */
        9 #include <string.h>
          
       10 int main()
       11 {
       12     int mytid;              /* our task ID          */
       13     int slave_tid;          /* task ID of the slave */
       14     int result;
       15     char message[] = "hello pvm";
       16     
       17     /* enroll ourselves into the PVM system and get our ID */
       18     mytid = pvm_mytid();
          
       19     /* spawn the slave */
       20     result = pvm_spawn("slave_pvm", (char**)0, PvmTaskDefault, 
       21                         "", 1, &slave_tid);
          
       22     /* check if the slave was spawned successfully          */
       23     if(result != 1)
       24     {
       25         fprintf(stderr, "Error: Cannot spawn slave.\n");
          
       26         /* clean up and exit from the PVM system            */
       27         pvm_exit();
       28         exit(EXIT_FAILURE);
       29     }
          
       30     /* initialize the data buffer to send data to slave     */
       31     pvm_initsend(PvmDataDefault);
          
       32     /* ``pack'' the string into the data buffer             */
       33     pvm_pkstr(message);
          
       34     /* send the string to the slave with a message tag of 0 */
       35     pvm_send(slave_tid, 0);
          
       36     /* wait and receive the result string from the slave    */
       37     pvm_recv(slave_tid, 0);
          
       38     
       39     /* ``unpack'' the result from the slave                 */
       40     pvm_upkstr(message);
          
       41     /* show the result from the slave                       */
       42     printf("Data from the slave : %s\n", message);
          
       43     /* clean up and exit from the PVM system                */
       44     pvm_exit();
       45     
       46     exit(EXIT_SUCCESS);
       47 } /* end main() */
          
       48 /* end master_pvm.c */
  

  ---

        1 /* -------------------------------------------------------------------- *
        2  * slave_pvm.c                                                          *
        3  *                                                                      *
        4  * This is the slave program for the simple PVM demonstration           *
        5  * -------------------------------------------------------------------- */
        6 #include <stdio.h>
        7 #include <ctype.h>
        8 #include <stdlib.h>
        9 #include <pvm3.h>
          
       10 #define MSG_LEN     20
       11 void convert_to_upper(char*);
          
       12 int main()
       13 {
       14     int mytid;
       15     int parent_tid;
       16     char message[MSG_LEN];
          
       17     /* enroll ourselves into the PVM system         */
       18     mytid = pvm_mytid();
          
       19     /* get the task ID of the master                */
       20     parent_tid = pvm_parent();
          
       21     /* receive the original string from master      */
       22     pvm_recv(parent_tid, 0);
       23     pvm_upkstr(message);
          
       24     /* convert the string to upper case             */
       25     convert_to_upper(message);
          
       26     /* send the converted string to the master      */
       27     pvm_initsend(PvmDataDefault);
          
       28     pvm_pkstr(message);
       29     pvm_send(parent_tid, 0);
          
       30     /* clean up and exit from the PVM system        */
       31     pvm_exit();
       32     
       33     exit(EXIT_SUCCESS);
       34 } /* end main() */
          
       35 /* function to convert the given string into upper case */
       36 void convert_to_upper(char* str)
       37 {
       38     while(*str != '\0')
       39     {
       40         *str = toupper(*str);
       41         str++;
       42     }
       43 } /* end convert_to_upper() */
          
       44 /* end slave_pvm.c */
  

  ---

        1 # Make file for the demo PVM program
          
        2 .SILENT :
        3 # paths fro PVM include files and libraries
        4 INCDIR=-I/usr/share/pvm3/include
        5 LIBDIR=-L/usr/share/pvm3/lib/LINUX
          
        6 # link the PVM library
        7 LIBS=-lpvm3
        8 CFLAGS=-Wall
        9 CC=gcc
       10 TARGET=all
          
       11 # this is where the PVM executables go
       12 PVM_HOME=$(HOME)/pvm3/bin/LINUX
          
       13 all : $(PVM_HOME)/master_pvm $(PVM_HOME)/slave_pvm
          
       14 $(PVM_HOME)/master_pvm : master_pvm.c
       15     $(CC) -o $(PVM_HOME)/master_pvm master_pvm.c $(CFLAGS) $(LIBS) \
       16           $(INCDIR) $(LIBDIR)
          
       17 $(PVM_HOME)/slave_pvm : slave_pvm.c
       18     $(CC) -o $(PVM_HOME)/slave_pvm slave_pvm.c $(CFLAGS) $(LIBS) \
       19           $(INCDIR) $(LIBDIR)
  

  ---

  Once your programs have been compiled, you must copy them into the ~/pvm3/bin/LINUX directory. (The makefile does it by default). Now to run the programs, you must first start the PVM system. To do this give the command pvm to start the PVM Console. Now at the pvm> prompt, type quit. The output will be as follows:

  pvm> quit
  quit
  
  Console: exit handler called
  pvmd still running.
  

  Notice the last line, indicating that the PVM daemon (pvmd) is still running. To run the PVM programs, you need to run the PVM daemon which manages the exchange of messages and that what we are doing here. Once the PVM daemon is running, you can run the program by the following commands:

  [rahul@joshicomp rahul]$ cd ~/pvm3/bin/LINUX/
  [rahul@joshicomp LINUX]$ ./master_pvm
  Data from the slave : HELLO PVM
  [rahul@joshicomp LINUX]$
  

  Notice that the string is now in upper case as expected.

  3.2 Explanation of the program

  In this section, we will see exactly how this program works. First of all to use PVM function, you need to include a header file pvm3.h in your programs. This is done in line 8 of master_pvm.c and in line 9 of slave_pvm.c. Also when compiling the programs, you need to link it with the PVM library. This is done by specifying the -lpvm3 option to the compiler, as done in line 7 of makefile.demo. Also, you need to specify to the compiler the paths of header and library files, as is done on lines 4 and 5 of the makefile.

  In the master program, we first get the task ID of the master by calling the PVM function pvm_mytid(). The PVM system assigns each process a unique 32 bit integer called as its task ID in the same way as Linux assigns each process a process ID. The task ID helps us identify the process with which we need to communicate. However, the master does not uses its task ID (stored in mytid) ever. Our intention here is just to call the function pvm_mytid(). This function enrolls the process into the PVM system and generates a unique task ID for the process. If we do not explicitly enroll the process, PVM automatically enrolls our process on the first call to any PVM function. Next we use pvm_spawn() to create the slave process. The first parameter, "slave_pvm" is the name of the executable for the slave. The second parameter is the arguments that you wish to the pass to the slaves (similar to argv in normal C). Since we do not want to send any arguments, we set this value to 0. The third parameter is a flag with which we can control how and where PVM starts the slave. Since we have only a single machine, we set this flag to PvmTaskDefault, specifying PVM to use default criteria while spawning the slave. The fourth parameter is the name of the host or the architecture on which we wish to run the program and here it is kept empty. It is used to specify the host or the architecture when the flag is other than PvmTaskDefault.The fifth parameter specifies the number of slaves to spawn and the sixth parameter is a pointer to an array in which the IDs of the slaves will be returned. This function returns the number of slaves actually spawned which we check for correctness.

  A message in PVM consists of basically two parts, the data and a tag that identifies the type of the message. The tag helps us distinguish between different messages. For example, in the addition example, which we are going to implement, suppose that you are expecting that each slave will send to the master an integer which is the sum of the elements it added. It is also quite possible that some slave may encounter some error and may want to send the master an integer which indicates the error code. How does the master distinguish whether an integer it received from the slave is an intermediate result or an error code? This is where tags come in picture. You may assign the message for intermediate result a tag say MSG_RESULT which you will #define in some header file and a tag say MSG_ERROR for the message indicating error. The master will then look at the message tags to decide whether the message contains intermediate result or error.

  To send a message, you first need to ``initialize'' the send buffer. This is done by calling the pvm_initsend() function. The parameter to this function specifies the ``encoding'' scheme to be used. When we want to exchange data between machines with different architectures (like say between a Pentium machine and a SPARC Workstation) then we need to encode the data at the sending end and decode at the receiving end so that data is properly delivered. The parameter to pvm_initsend() specifies the encoding scheme to be used. The value PvmDataDefault specifies an encoding scheme which enables data to be safely exchanged between heterogeneous architectures. Once the buffer has been initialized, we need to put data into the buffer and encode it. In our case, the data is a string, so we use the function pvm_pkstr() to ``pack'' i.e. encode and put the data into the buffer. If we had to send an integer, there is a different function pvm_pkint(). Similarly, there are functions for other data types. Once the data is packed, we call pvm_send() to send the message. The first argument is the ID of the process to which the message is to be sent and the second argument is the message tag. Since there is only one type of message here, we set the tag to 0.

  Once the data is sent to the slave, the slave will process it and return it to the master as we shall see. So we now call pvm_recv() to receive the data from the slave. Again, the parameters are the task ID from which the message is expected and the tag of the expected message. If the desired message has not yet been sent, this function waits and does not return. Thus, in effect, the master is now waiting for the slave to process the data. Once the message arrives, the data is still in the receive buffer. It needs to be ``unpacked'' i.e decoded to get the original message. This decoding is done by the pvm_upkstr() function. We then display the processes string.

  Before the PVM program exits, it must tell the PVM system that it is leaving the PVM system so that resources occupied by the process can be released. This is done by calling the pvm_exit() function. After that, the master exits.

  The slave program is easy to understand. First it finds the task ID of the master (which is also its parent as the master spawned the slave) by calling the function pvm_parent(). It then receives the message string from the master, converts it to uppercase and send the resulting string to the master.

  3.3 The Addition Program

  Now that you know some basics of a PVM program, let us implement the addition algorithm we developed using PVM. There will be one master and 4 slaves. The master will first spawn 4 slaves and send each one their part of data. The slaves will add the data and send the results to the master. Thus, two types of messages are exchanged, one when the master send data to slaves, for which we will use the tag MSG_DATA and the other when the slaves send results to master, for which we will use the tag MSG_RESULT. The rest is simple. The master and the slave programs are given below.

  ---

        1 /* -------------------------------------------------------------------- *
        2  * common.h                                                             *
        3  *                                                                      *
        4  * This header file defines some common constants.                      *
        5  * -------------------------------------------------------------------- */
        6 #ifndef COMMON_H
        7 #define COMMON_H
      
        8 #define NUM_SLAVES      4                   /* number of slaves     */
        9 #define SIZE            100                 /* size of total data   */
       10 #define DATA_SIZE       (SIZE/NUM_SLAVES)   /* size for each slave  */
      
       11 #endif
       12 /* end common.h */
  

  ---

        1 /* -------------------------------------------------------------------- *
        2  * tags.h                                                               *
        3  *                                                                      *
        4  * This header file defines the tags that will be used for messages.    *
        5  * -------------------------------------------------------------------- */
        6 #ifndef TAGS_H
        7 #define TAGS_H
      
        8 #define MSG_DATA            101     /* data from master to slave    */
        9 #define MSG_RESULT          102     /* result from slave to master  */
      
       10 #endif
      
       11 /* end tags.h */
  

  ---

    1 /* -------------------------------------------------------------------- *
    2  * master_add.c                                                         *
    3  *                                                                      *
    4  * Master program for adding the elements of an array by using PVM      *
    5  * -------------------------------------------------------------------- */
    6 #include <stdio.h>
    7 #include <stdlib.h>
    8 #include <pvm3.h>           /* PVM constants and declarations   */
    9 #include "tags.h"           /* tags for messages                */
   10 #include "common.h"         /* common constants                 */
      
   11 int get_slave_no(int*, int);
      
   12 int main()
   13 {
   14     int mytid;
   15     int slaves[NUM_SLAVES]; /* array to store the task IDs of slaves    */
   16     int items[SIZE];        /* data to be processes                     */
   17     int result, i, sum;
   18     int results[NUM_SLAVES];    /* results from the slaves              */
      
   19     /* enroll into the PVM system   */
   20     mytid = pvm_mytid();
      
   21     /* initialize the array `items' */
   22     for(i = 0; i < SIZE; i++)
   23         items[i] = i;
      
   24     /* spawn the slaves             */
   25     result = pvm_spawn("slave_add", (char**)0, PvmTaskDefault,
   26                        "", NUM_SLAVES, slaves);
      
   27     /* check if proper number of slaves are spawned     */
   28     if(result != NUM_SLAVES)
   29     {
   30         fprintf(stderr, "Error: Cannot spawn slaves.\n");
   31         pvm_exit();
   32         exit(EXIT_FAILURE);
   33     }
      
   34     /* distribute the data among the slaves     */
   35     for(i = 0; i < NUM_SLAVES; i++)
   36     {
   37         pvm_initsend(PvmDataDefault);
   38         pvm_pkint(items + i*DATA_SIZE, DATA_SIZE, 1);
   39         pvm_send(slaves[i], MSG_DATA);
   40     }
      
   41     /* receive the results from the slaves      */
   42     for(i = 0; i < NUM_SLAVES; i++)
   43     {
   44         int bufid, bytes, type, source;
   45         int slave_no;
   46         
   47         /* receive message from any of the slaves       */
   48         bufid = pvm_recv(-1, MSG_RESULT);
      
   49         /* get information about the message            */
   50         pvm_bufinfo(bufid, &bytes, &type, &source);
   51         
   52         /* get the slave number that sent the message   */
   53         slave_no = get_slave_no(slaves, source);
      
   54         /* unpack the results at appropriate position   */
   55         pvm_upkint(results + slave_no, 1, 1);
   56     }
      
   57     /* find the final result            */
   58     sum = 0;
   59     for(i = 0; i < NUM_SLAVES; i++)
   60         sum += results[i];
      
   61     printf("The sum is %d\n", sum);
      
   62     /* clean up and exit from the PVM system    */
   63     pvm_exit();
      
   64     exit(EXIT_SUCCESS);
   65 } /* end main() */
   66         
   67 /* function to return the slave number of a slave given its task ID */
   68 int get_slave_no(int* slaves, int task_id)
   69 {
   70     int i;
      
   71     for(i = 0; i < NUM_SLAVES; i++)
   72         if(slaves[i] == task_id)
   73             return i;
      
   74     return -1;
   75 } /* end get_slave_no() */
      
   76 /* end master_add.c */
  
  

  ---

    1 /* -------------------------------------------------------------------- *
    2  * slave_add.c                                                          *
    3  *                                                                      *
    4  * Slave program for adding elements of an array using PVM              *
    5  * -------------------------------------------------------------------- */
    6 #include <stdlib.h>
    7 #include <pvm3.h>
    8 #include "tags.h"
    9 #include "common.h"
      
   10 int main()
   11 {
   12     int mytid, parent_tid;
   13     int items[DATA_SIZE];           /* data sent by the master  */
   14     int sum, i;
   15     
   16     /* enroll into the PVM system       */
   17     mytid = pvm_mytid();
      
   18     /* get the task ID of the master    */
   19     parent_tid = pvm_parent();
      
   20     /* receive the data from the master */
   21     pvm_recv(parent_tid, MSG_DATA);
   22     pvm_upkint(items, DATA_SIZE, 1);
      
   23     /* find the sum of the elements     */
   24     sum = 0;
   25     for(i = 0; i < DATA_SIZE; i++)
   26         sum = sum + items[i];
      
   27     /* send the result to the master    */
   28     pvm_initsend(PvmDataDefault);
   29     pvm_pkint(&sum, 1, 1);
   30     pvm_send(parent_tid, MSG_RESULT);
      
   31     /* clean up and exit from PVM       */
   32     pvm_exit();
   33     
   34     exit(EXIT_SUCCESS);
   35 } /* end main() */
  
  

  ---

    1 # Make file for the PVM program for addition - makefile.add
      
    2 .SILENT :
    3 # paths fro PVM include files and libraries
    4 INCDIR=-I/usr/share/pvm3/include
    5 LIBDIR=-L/usr/share/pvm3/lib/LINUX
      
    6 # link the PVM library
    7 LIBS=-lpvm3
    8 CFLAGS=-Wall
    9 CC=gcc
   10 TARGET=all
      
   11 # this is where the PVM executables go
   12 PVM_HOME=$(HOME)/pvm3/bin/LINUX
      
   13 all : $(PVM_HOME)/master_add $(PVM_HOME)/slave_add
      
   14 $(PVM_HOME)/master_add : master_add.c common.h tags.h
   15     $(CC) -o $(PVM_HOME)/master_add master_add.c $(CFLAGS) $(LIBS) \
   16           $(INCDIR) $(LIBDIR)
   17   
   18 $(PVM_HOME)/slave_add : slave_add.c common.h tags.h
   19     $(CC) -o $(PVM_HOME)/slave_add slave_add.c $(CFLAGS) $(LIBS) \
   20          $(INCDIR) $(LIBDIR)
  

  ---

  Let us consider the slave program first, because it is simple. The slave receives the 25 array elements from the master in the array items, finds their sum and sends the result to the master with the message tag as MSG_RESULT. Now consider the master. We define an array slaves of size NUM_SLAVES which will store the task ID's of the slaves spawned by the parent. There is another array results in which the results from the slaves are stored. The master first initializes the array items and then spawns the slaves. After that it distributes the data among the slaves. In the call to pvm_pkint() on line 38, the first parameter is the pointer to the array in which the integers are stored, the second is the number of integers to pack and the third is the ``stride.'' Stride means how many elements to skip when packing. When it is 1, consecutive elements are packed. When it is 2, PVM will skip 2 elements when packing with the result that all even numbered elements (0, 2, 4 ...) will be packed. Here we keep its value as 1.

  Once the data has been distributed among the slaves, the master has to wait till the slaves return the intermediate results. One possibility when accepting the results is that the master will first collect the result from slave 0 (i.e slave whose task ID is stored in slave[0]), then from slave 1 and so on. However, this may not be an efficient approach. For example, it may be that slave 0 is working on a slower machine than slaves 1, 2 and 3. In that case, since the master is waiting from slave 0, the results from slaves 1, 2 and 3 are yet to be collected even though the calculations are completed. In this case it may be fine, but consider the situation in which the slave, when finished doing one job is given another job. In that case, we would like to give a slave its next job immediately after it has completed its current job. Thus, the master must be in a position to respond messages from any of the slaves. This is what is being done here.

  In the call to pvm_recv() on line 48, we know that the first parameter is the task ID of the message source. If this value is kept -1, it signifies a wild card i.e. messages from any process with message tag MSG_RESULT will be received by the master. The received message along with some control information is stored in a buffer called as active receive buffer. The call returns a unique ID for this buffer. Now, we want to know who is the sender of the message so that we can assign the message data to the appropriate element of the array results. The function pvm_bufinfo() returns information about the message in the buffer, such as the message tag, the number of bytes and the senders task ID. Once we have the senders task ID, we set the appropriate element of the results array to the integer sent by the slave. The rest of the program should be easy to understand.

  3.4 Working with PVM

  In case you are interested, you can think of some problems for which you can write parallel programs. Many a times, due to bugs etc., you may need to clean up the state of the things before starting. The PVM Console provides with the command halt that kills the PVM daemon. Then all the PVM processes will halt or you can halt them with the Linux kill command. In case you have a network of Linux machines interconnected by say a LAN, then you can also do ``real'' parallel processing. First of all, install PVM on all the hosts you wish to use and then use the add command in the PVM Console to add hosts to the virtual machine. Then PVM will schedule some of the processes to run on these hosts, so that real parallel processing is achieved.

  4. Implementing with MPI

  We have seen in the previous section the implementation of the addition program using the PVM. Now let us consider another approach that can be used in developing parallel programs. This approach is using the MPI library. MPI stands for Message Passing Interface. It is a standard developed to enable us to write portable message passing applications. It provides functions for exchanging messages and many other activities as well. It must be noted that unlike PVM which is a software system, MPI is a standard, so that many implementations of the MPI standard exist. We will use an implementation of MPI called LAM which stands for Local Area Multicomputer. It is also available on the Red Hat Linux CD as an RPM package, so installation may not be a problem.

  After you have installed the RPM package, go to the /usr/boot directory and create a file named conf.lam and type in a single line in it: lamd $inet_topo. The same directory will also have a file named bhost.def else create it and type in a single line in it: localhost. Now to test whether everything is working correctly, type at the prompt, lamboot. You will get the following response:

  [rahul@joshicomp boot]$ lamboot
  
  LAM 6.3.1/MPI 2 C++/ROMIO - University of Notre Dame
  
  [rahul@joshicomp boot]$
  

  If the output indicates an error, then there is some problem with the installation, either follow the above steps or see the lamboot(1) manual page for troubleshooting.

  Assuming that LAM/MPI is properly installed on your system, let us again write a small demonstration program for MPI.

  4.1 A Demonstration MPI Program

  We will again write a simple master - slave program in which we are supposed to evaluate the expression (a + b) * (c - d). The master will read the values of a, b, c, and d from the user and one slave will calculate (a + b) and the other one will calculate (c - d). The program is as follows.

  ---

    1 /* -------------------------------------------------------------------- *
    2  * mpi_demo.c                                                           *
    3  *                                                                      *
    4  * A simple MPI demonstration program to evaluate an expression.        *
    5  * -------------------------------------------------------------------- */
    6 #include <stdio.h>
    7 #include <stdlib.h>
    8 #include <lam/mpi.h>            /* for MPI constants and functions      */
      
    9 #define MSG_DATA        100     /* message from master to slaves        */
   10 #define MSG_RESULT      101     /* message from slave to master         */
      
   11 #define MASTER          0       /* rank of master                       */
   12 #define SLAVE_1         1       /* rank of first slave                  */
   13 #define SLAVE_2         2       /* rank of second slave                 */
      
   14 /* functions to handle the tasks of master, and the two slaves          */
   15 void master(void);
   16 void slave_1(void);
   17 void slave_2(void);
      
   18 int main(int argc, char** argv)
   19 {
   20     int myrank, size;
   21     
   22     /* initialize the MPI system                                        */
   23     MPI_Init(&argc, &argv);
      
   24     /* get the size of the communicator i.e. number of processes        */
   25     MPI_Comm_size(MPI_COMM_WORLD, &size);
      
   26     /* check for proper number of processes                             */
   27     if(size != 3)
   28     {
   29         fprintf(stderr, "Error: Three copies of the program should be run.\n");
   30         MPI_Finalize();
   31         exit(EXIT_FAILURE);
   32     }
   33     
   34     /* get the rank of the process                                      */
   35     MPI_Comm_rank(MPI_COMM_WORLD, &myrank);
      
   36     /* perform the tasks according to the rank                          */
   37     if(myrank == MASTER)
   38         master();
   39     else if(myrank == SLAVE_1)
   40         slave_1();
   41     else
   42         slave_2();
      
   43     /* clean up and exit from the MPI system                            */
   44     MPI_Finalize();
      
   45     exit(EXIT_SUCCESS);
   46 } /* end main() */
      
   47 /* function to carry out the masters tasks          */
   48 void master(void)
   49 {
   50     int a, b, c, d;
   51     int buf[2];
   52     int result1, result2;
   53     MPI_Status status;
      
   54     printf("Enter the values of a, b, c, and d: ");
   55     scanf("%d %d %d %d", &a, &b, &c, &d);
      
   56     /* send a and b to the first slave              */
   57     buf[0] = a;
   58     buf[1] = b;
   59     MPI_Send(buf, 2, MPI_INT, SLAVE_1, MSG_DATA, MPI_COMM_WORLD);
      
   60     /* send c and d to the secons slave             */
   61     buf[0] = c;
   62     buf[1] = d;
   63     MPI_Send(buf, 2, MPI_INT, SLAVE_2, MSG_DATA, MPI_COMM_WORLD);
      
   64     /* receive results from the slaves              */
   65     MPI_Recv(&result1, 1, MPI_INT, SLAVE_1, MSG_RESULT, 
   66              MPI_COMM_WORLD, &status);
   67     MPI_Recv(&result2, 1, MPI_INT, SLAVE_2, MSG_RESULT, 
   68              MPI_COMM_WORLD, &status);
      
   69     /* final result                                 */
   70     printf("Value of (a + b) * (c - d) is %d\n", result1 * result2);
   71 } /* end master() */
      
   72 /* function to carry out the tasks of the first slave       */
   73 void slave_1(void)
   74 {
   75     int buf[2];
   76     int result;
   77     MPI_Status status;
   78     
   79     /* receive the two values from the master       */ 
   80     MPI_Recv(buf, 2, MPI_INT, MASTER, MSG_DATA, MPI_COMM_WORLD, &status);
   81     
   82     /* find a + b                                   */
   83     result = buf[0] + buf[1];
      
   84     /* send result to the master                    */
   85     MPI_Send(&result, 1, MPI_INT, MASTER, MSG_RESULT, MPI_COMM_WORLD);
   86 } /* end slave_1() */
      
   87 /* function to carry out the tasks of the second slave      */
   88 void slave_2(void)
   89 {
   90     int buf[2];
   91     int result;
   92     MPI_Status status;
   93     
   94     /* receive the two values from the master       */
   95     MPI_Recv(buf, 2, MPI_INT, MASTER, MSG_DATA, MPI_COMM_WORLD, &status);
   96     
   97     /* find c - d                                   */
   98     result = buf[0] - buf[1];
      
   99     /* send result to master                        */
  100     MPI_Send(&result, 1, MPI_INT, MASTER, MSG_RESULT, MPI_COMM_WORLD);
  101 } /* end slave_2() */
      
  102 /* end mpi_demo.c */
  

  ---

    1 # Makefile for MPI demo program - makefile.mpidemo
    2 .SILENT:
    3 CFLAGS=-I/usr/include/lam -L/usr/lib/lam
    4 CC=mpicc
      
    5 mpi_demo : mpi_demo.c
    6     $(CC) $(CFLAGS) mpi_demo.c -o mpi_demo
  

  ---

  To compile this program, give the command make -f makefile.mpidemo. Once you have compiled the program, to run the program you first need to ``start'' or ``boot'' the Local Area Multicomputer system. This is done with the lamboot command. After that, to run the program by giving the following command: mpirun -np 3 mpi_demo.

  [rahul@joshicomp parallel]$ lamboot
  
  LAM 6.3.1/MPI 2 C++/ROMIO - University of Notre Dame
  
  [rahul@joshicomp parallel]$ mpirun -np 3 mpi_demo
  Enter the values of a, b, c, and d: 1 2 3 4
  Value of (a + b) * (c - d) is -3
  [rahul@joshicomp parallel]$
  

  4.2 Explanation of the Program

  To use the MPI system and functions, you first need to include the header file mpi.h as is done in line 8. In case of PVM, different processes are identified with their task ID's. In case of MPI, the MPI system assigns each process a unique integer called as its rank beginning with 0. The rank is used to identify a process and communicate with it. Secondly, each process is a member of some communicator. A communicator can be thought of as a group of processes that may exchange messages with each other. By default, every process is a member of the communicator called MPI_COMM_WORLD. Although we can create new communicators, this leads to an unnecessary increase in complexity, so we suffice ourselves by using the MPI_COMM_WORLD communicator.

  Any MPI program must first call the MPI_Init() function. This function is used by the process to enter the MPI system and also do any specific initialization required by the system. Next, we get the size of the MPI_COMM_WORLD communicator i.e. the number of processes in it using the MPI_Comm_size() function. The first parameter is the communicator and the second is a pointer to an integer in which the size will be returned. Here, we need exactly 3 processes, one master and two slaves. After that, we get the rank by calling MPI_Comm_rank(). The three processes will have ranks 0, 1 and 2. All these processes are essentially identical i.e. there is no inherent master - slave relationship between them. So it is up to us to decide who will be the master and who will be the slaves. We choose rank 0 as master and ranks 1 and 2 as slaves. It can also be seen that we have included the code for both the master and the two slaves in the same program. Depending upon the rank, we choose to execute the appropriate function. Note that there is no spawning of processes as in PVM, and as we shall see, we choose to decide the number of process to be spawned from a command line argument rather than the program spawning slaves. Once the execution is finished, we must call the MPI_Finalize() function to perform final clean up.

  Let us now consider the master function. After reading the values of a, b, c, and d from the user, the master must send a and b to slave 1 and c and d to slave 2. Instead of sending the variables individually, we choose to pack them up in an array and send the array of 2 integers instead. It is always better to pack up the data you want to send into a single message rather than to send a number of messages for individual data items, this saves the communication overhead involved in passing the messages. Once the buffer is ready, unlike PVM, we do not need to pack or encode the data, MPI will manage these details internally. So we can directly call the MPI_Send() function to send the data. The first parameter (line 59) is the address of the buffer, the second one the number of elements in the message, the third is a specification of the data type of the buffer, which here is MPI_INT specifying that the buffer is an array of integers. Next comes the rank of the process to which we want to send the message. Here it is SLAVE_1 (#defined as 1). Next is the message tag similar to that in case of PVM. Final parameter is the communicator of which the receiver is a member, which in this case, is MPI_COMM_WORLD.

  Once the data is distributed among the slaves, the master must wait for the slaves to send the results. For simplicity, we first collect the message from the slave 1 and then from slave 2. To receive a message, we use the MPI_Recv() function. Again, packing and decoding is handled by MPI internally. The first argument (line 65) is the address of the buffer in which to receive the data. The second is the size of the buffer in terms of the number of elements, which in this case is 1. Next is the data type, which is MPI_INT here. Next three parameters specify the rank of the source of the message, the tag of the expected message and the communicator of which the source is the member. The final argument is a pointer to a structure of type MPI_Status in which some status information will be returned (however, we ignore this information). Now that you know about the basic MPI terms, the slave_1() and slave_2() functions should be clear.

  In this program, the code for the master as well as the slaves was in the same executable file. Later on we will see how we can execute multiple executables. From the makefile, we see that to compile the MPI program, a wrapper program mpicc is provided which links the required libraries automatically. To run the program, use the mpirun -np 3 mpi_demo command after booting the LAM. Here we specify LAM to create 3 processes, one master and two slaves.

  4.3 The Addition Program Again

  Let us now re implement the addition program that we designed before using MPI. Here we will also show you how to execute separate programs in MPI. When we use a single executable in the MPI program, we call it Single Program Multiple Data (SPMD) application. When two or more executables are involved, we call it Multiple Program Multiple Data (MPMD) application. With LAM, MPMD programs are executed with the help of an application schema. But before that, let us see the source of the master and the slave programs.

  ---

    1 /* -------------------------------------------------------------------- *
    2  * master_mpi.c                                                         *
    3  *                                                                      *
    4  * Master program for adding the elements of an array using MPI         *
    5  * -------------------------------------------------------------------- */
    6 #include <stdio.h>
    7 #include <stdlib.h>
    8 #include <lam/mpi.h>        /* MPI constants and functions              */
    9 #include "tags.h"           /* tags for different messages              */
   10 #include "common.h"         /* common constants                         */
      
   11 int main(int argc, char** argv)
   12 {
   13     int size, i, sum;
   14     int items[SIZE];
   15     int results[NUM_SLAVES];
   16     MPI_Status status;
      
   17     /* initlalize the MPI System                */
   18     MPI_Init(&argc, &argv);
      
   19     /* check for proper number of processes     */
   20     MPI_Comm_size(MPI_COMM_WORLD, &size);
      
   21     if(size != 5)
   22     {
   23         fprintf(stderr, "Error: Need exactly five processes.\n");
   24         MPI_Finalize();
   25         exit(EXIT_FAILURE);
   26     }
      
   27     /* initialize the `items' array             */
   28     for(i = 0; i < SIZE; i++)
   29         items[i] = i;
      
   30     /* distribute the data among the slaves     */
   31     for(i = 0; i < NUM_SLAVES; i++)
   32         MPI_Send(items + i*DATA_SIZE, DATA_SIZE, MPI_INT, i + 1,
   33                  MSG_DATA, MPI_COMM_WORLD);
      
   34     /* collect the results from the slaves      */
   35     for(i = 0; i < NUM_SLAVES; i++)
   36     {
   37         int result;
   38         
   39         MPI_Recv(&result, 1, MPI_INT, MPI_ANY_SOURCE, MSG_RESULT,
   40                  MPI_COMM_WORLD, &status);
   41         results[status.MPI_SOURCE - 1] = result;
   42     }
      
   43     /* find the final answer                    */
   44     sum = 0;
   45     for(i = 0; i < NUM_SLAVES; i++)
   46         sum = sum + results[i];
      
   47     printf("The sum is %d\n", sum);
      
   48     /* clean up and exit the MPI system         */
   49     MPI_Finalize();
      
   50     exit(EXIT_SUCCESS);
   51 } /* and main() */
      
   52 /* end master_mpi.c */
  

  ---

    1 /* -------------------------------------------------------------------- *
    2  * slave_mpi.c                                                          *
    3  *                                                                      *
    4  * Slave program for adding array elements using MPI.                   *
    5  * -------------------------------------------------------------------- */
    6 #include <stdio.h>
    7 #include <stdlib.h>
    8 #include <lam/mpi.h>        /* MPI functions and constants  */
    9 #include "tags.h"           /* message tags                 */
   10 #include "common.h"         /* common constants             */
      
   11 #define MASTER  0           /* rank of the master           */
      
   12 int main(int argc, char** argv)
   13 {
   14     int items[DATA_SIZE];
   15     int size, sum, i;
   16     MPI_Status status;
      
   17     /* initialize the MPI system            */
   18     MPI_Init(&argc, &argv);
      
   19     /* check for proper number of processes */
   20     MPI_Comm_size(MPI_COMM_WORLD, &size);
      
   21     if(size != 5)
   22     {
   23         fprintf(stderr, "Error: Need exactly five processes.\n");
   24         MPI_Finalize();
   25         exit(EXIT_FAILURE);
   26     }
      
   27     /* receive data from the master         */
   28     MPI_Recv(items, DATA_SIZE, MPI_INT, MASTER, MSG_DATA,
   29              MPI_COMM_WORLD, &status);
      
   30     /* find the sum                         */
   31     sum = 0;
   32     for(i = 0; i < DATA_SIZE; i++)
   33         sum = sum + items[i];
      
   34     /* send the result to the master        */
   35     MPI_Send(&sum, 1, MPI_INT, MASTER, MSG_RESULT, MPI_COMM_WORLD);
      
   36     /* clean up and exit MPI system         */
   37     MPI_Finalize();
      
   38     exit(EXIT_SUCCESS);
   39 } /* end main() */
      
   40 /* end slave_mpi.c */
  

  ---

    1 # Makefile for MPI addition program - makefile.mpiadd
    2 .SILENT:
    3 CFLAGS=-I/usr/include/lam  -L/usr/lib/lam
    4 CC=mpicc
      
    5 all : master_mpi slave_mpi
      
    6 master_mpi : master_mpi.c common.h tags.h
    7     $(CC) $(CFLAGS) master_mpi.c -o master_mpi
      
    8 slave_mpi : slave_mpi.c common.h tags.h
    9     $(CC) $(CFLAGS) slave_mpi.c -o slave_mpi
  

  ---

  To compile the programs, type make -f makefile.mpiadd. (The files common.h and tags.h are the same as used for the PVM program.) This will create the master_mpi and slave_mpi executables. Now how do we tell MPI to run both these executables. This is where application schema file comes in. The application schema file specifies the executables to be run, the nodes on which to run and the number of copies of the executable to run. Create a new file add.schema and type in it the following lines:

  # Application schema for the addition program using MPI
  n0 master_mpi
  n0 -np 4 slave_mpi
  

  This file specifies that MPI should start 1 copy of the master (which will have rank 0) and 4 copies of slaves on the node n0, i.e. the local node. You can specify many more parameters in this schema file like command line arguments etc., see the manual page appschema(1). Once the schema file is ready, you can run the programs as follows:

  [rahul@joshicomp parallel]$ lamboot
  
  LAM 6.3.1/MPI 2 C++/ROMIO - University of Notre Dame
  
  [rahul@joshicomp parallel]$ mpirun add.schema
  The sum is 4950
  [rahul@joshicomp parallel]$
  

  Much of the program should be easy to understand. On line 39, when receiving intermediate results from the slaves, we specify the source as MPI_ANY_SOURCE, since we want to respond to slaves in the order in which they complete the calculations, as discussed earlier. In this case, the status structure contains the actual source in the field MPI_SOURCE. We use this information to set the appropriate element from the results array to the intermediate result received.

  In case you have a network of interconnected computers, you can make programs run on many computers by suitably modifying the application schema file. Instead of specifying n0 as the host, specify the name of the host and the number of processes you wish to schedule on that host. For more information about this, see the manual pages and the references.

  5. Conclusion

  We have seen how to write parallel programs using the PVM and MPI libraries. Since there libraries are available on many platforms and these are the defacto standards used for implementing parallel programs, programs written with PVM or MPI will run with little or no modification on large scale machines, if the need arises. What we have basically concentrated on in this article is the point to point communication functions provides by these libraries and their use in message passing. Apart from these facilities, both PVM and MPI provide a number of advanced features such as collective communication (broadcasting or multicasting), process groups and group management, reduction functions etc. You are welcome to explore these advanced features. These public domain softwares enable us to use a network of computers as a single large computer, so in case you have some such large problem to solve, you may consider using a network at your college or office. You will have to refer to the books given below for the exact details of how such a setup may be established. Many tutorials as well as books are available to help you. Below is a list of the material I referred.

  1. PVM: Parallel Virtual Machine - A User's Guide and Tutorial for Networked Parallel Computing, Al Geist, Adam Beguelin, Jack Dongarra, Robert Manchek, Weicheng Jiang and Vaidy Sunderam, MIT Press. Available at www.netlib.org
  2. MPI: The Complete Reference, Marc Snir, Steve Otto, Steven Huss-Lederman, David Waker and Jack Dongarra, MIT Press. Available at www.netlib.org.
  3. RS/6000 SP: Practical MPI Programming,Yukiya Aoyama and Jan Nakano, International Techical Support Organization, IBM Corporation, www.redbooks.ibm.com.
  4. A Beginner's Guide to PVM Parallel Virtual Machine, Clay Breshears and Asim YarKhan, Joint Institute of Computational Science, University of Tennessee, USA. www-jics.cs.utk.edu/PVM/pvm/_guide.html.
  5. PVM: An Introduction to Parallel Virtual Machine, Emily Angerer Crawford, Office of Information Technology, High Performance Computing, www.hpc.gatech.edu/seminar/pvm.html.

  6. Acknowlegements

  I would like to thank my project guide Dr. Uday Khedker for his encouragement and help. I would like to thank the Center for Developement of Advanced Computing for allowing me to run the MPI and PVM programs on the PARAM Supercomputer and Dr. Anabarsu for guiding me during the implementation.

  ---

  Copyright © 2001, Rahul U. Joshi.
  Copying license http://www.linuxgazette.com/copying.html
  Published in Issue 65 of Linux Gazette, April 2001

  "Linux Gazette...making Linux just a little more fun!"

  ---

  Web Portals Bank on Open-Source Infrastructure

  By

  ---

  Wireless Developer Network (WDN) and GeoCommunity say sayonara to the database big boys.

  When a single company formed to operate two web portals for online communities, they turned to the biggest names in the business to build their technology infrastructure. Wireless Developer Network (WDN) for wireless communications professionals, and its sister site, GeoCommunity for geographic information specialists, used hosting services to license the use of Microsoft and Oracle respectively. The two sites were up and running in no time, offering virtual homes to thousands of professionals who needed industry news, software downloads, product reviews and live chats. But while the portals seemed a success on the surface, disaster lurked not far below.

  Within months, WDN ran into performance problems with SQL Server and security issues with MS IIS. Oracle 8i worked like a dream for the GeoCommunity, but the licensing fees threatened to crush the small company. As the portal expanded capacity and users, it knew Oracle's aggressive pricing structures would cut even deeper into its slim profit margin. That's when the company's technical staff began to push a radical concept: ditch the big boys in favor of a single open-source technology platform for both web portals.

  The web portals had to be able to serve web pages up 24/7 without any crashes or service interruptions. The developers wanted to go with Linux because their experiences told them it was a stable and reliable platform for the web. They also knew firsthand that Apache web servers were a superior product, faster, more scalable and easier to configure, and optimized for the Linux platform.

  The technical staff had worked in both open-source and proprietary environments, and had come to believe open source was the more secure choice for web-based applications. In their view, open-source technologies grew up on the web, while most proprietary applications were later adapted to it. Throughout the open-source development process, developers drill down on security and performance issues for web applications. With hundreds of developers and users testing and tweaking the programs, security holes are often caught and corrected at an early stage of the development process.

  WDN's leadership quickly bought into the idea. However, many managers believe that "nothing good can be free" and subscribe to the common myth that open-source products lack professional technical support. But this company's senior management team was quickly sold on their developers' enthusiasm and positive experiences with open source.

  In fact, managers skittish about defying convention need only look around for evidence of the proliferation of open-source applications and tools in business. Linux powers an estimated 36 percent of Internet-connected servers today, while Apache web servers are on about 61 percent of public web sites, according to the Internet research firm, Netcraft. Industry analysts at Forrester Research recently identified open source as a powerful growing trend in business with potential to radically reshape the software industry by 2004.

  Yet the database market, the core of web-based businesses today, still remains firmly in the grip of proprietary vendors such as Oracle, Microsoft and IBM. But in recent years, open-source databases such as PostgreSQL and MySQL have evolved to the point where they're beginning to compete with the propriety giants in performance and functionality. They're attracting skilled development and user communities, as well as enterprise business users across a wide range of industries, and are rising up to challenge the proprietary status quo in the competitive database market.

  The heart of the portals

  For WDN and GeoCommunity, the most difficult decision related to its technology infrastructure was its choice of the right database management system. The portals needed a system that was scalable and functional enough to handle thousands of visitors each month and power scores of dynamic applications, including e-commerce.

  The portals tested two of the most widely used open source databases, MySQL and PostgreSQL. While MySQL was simple to configure and use, it lacked the transaction support and scalability that the company needed to run their highly interactive sites. MySQL has attracted a large user base, but the staff thought it seemed more suitable for lower-traffic web sites. The staff also ran rigorous tests on PostgreSQL, a heavy-duty object-relational database. It withstood the barrage of tests without flinching, supporting advanced features during heavy simulated transactions very well.

  After downloading their selection of open-source applications, including the Red Hat Linux operating system, Apache web server and PostgreSQL database, the technical staff configured the system in less than a half hour. The portals run 12 servers, with PostgreSQL powering dynamic applications such as book sales, message boards and mailing lists. The new system was up and running in minutes, with no interruptions, and neither web portal has since crashed or lost data.

  Confronting the FUD (fear, uncertainty and doubt) factor

  Yet there are good reasons why open-source technologies were once the exclusive domain of skilled hackers and expert users. In the past, these applications purposefully lacked the bells and whistles of their proprietary competition and were difficult for the less advanced user even to install. Open-source programs have become much more user-friendly over time because independent developers have begun to pay greater attention to improving tools, additional features and perhaps most importantly, documentation.

  For WDN and GeoCommunity, the decision to migrate from a proprietary to an open-source system was less difficult than for most traditional businesses. The web portals employ technical staff with experience in both environments. At every level, the company embraced the idea of adopting a more flexible, and less financially draining, open-source alternative. They understood the open-source development model and bought into its underlying philosophy. Just as importantly, they possessed the technical skills to confront many of the issues that could arise in an open-source platform. In fact, with access to their new system's code, they could now even modify their software's features to better fit the company's needs.

  Many e-businesses like these web portals, along with brick and mortar retailers that are moving into e-commerce, have similar needs, but lack the background and technical expertise to easily integrate open-source technologies or migrate to a fully open-source platform. These businesses simply want web sites that their customers and vendors can use without difficulty. They need database-enabled web applications with 24/7 availability that won't crash or lose data--even with thousands of daily transactions. They want a site that always works, convenient ways for customers to buy their products, and secure methods through which to bring in their money. Because database applications are so critical to their mission, many businesses adopt well-known proprietary systems, feeling confident these companies will deliver quality and reliability. Yet the rising costs, the uncertain economy, and in some cases, the surprisingly unpredictable performance of commercial applications, all have begun to spark greater interest in open-source technologies today.

  Still, these businesses are understandably skeptical about open source. They're used to the proprietary business model, and can't quite fathom why good software applications would be available to download for free from the Internet. The fact that these applications are not owned by a corporation causes suspicion and concern; if no single vendor owns it, people assume the software is not secure, powerful or reliable, and that it lacks accessible support and services. And the idea of thousands of independent developers around the world collaborating to create free software strikes many business managers as chaotic, which makes them even more reluctant to trust the results.

  Slowly, these businesses are becoming educated about the open-source development model, which evolved not to make money, but to produce functional software efficiently. They're finding that while the development process varies for each open-source application today, the best projects most often attract a global community of highly skilled developers. And it's becoming clearer that these systematic meritocracies encourage rigorous testing and rapid development rates, and result in fewer bugs and security holes and more frequent releases of new and improved features.

  A growing number of e-businesses such as WDN and the GeoCommunity are building their businesses on open-source platforms. These web portals have found that the software's fast-evolving development cycles, its lower costs, and its customizability make it ideal in their high-growth, quick-changing industries. The lower overall cost of open-source software is attractive to small and mid-sized businesses like these, who often have to spend thousands--even hundreds of thousands--of dollars on purchasing or licensing proprietary applications alone.

  Another important advantage is that its code is open and modifiable. The open-source model rests on the belief that software develops faster and better when its source code is accessible to all skilled developers. Mature open-source technologies such as Apache, Linux, and PostgreSQL have thrived under the principles of open collaboration. Similarly, businesses that employ open source technologies can benefit both from open source's accelerated development model and free access to its internal code, which enables them to modify the code as needed. Open-source technology is highly conducive to innovation, and ensures that most of the applications it produces improve continuously and quickly. Companies that use it usually find that their software programs evolve as quickly as their businesses do.

  Why isn't everyone using open source?

  The perceived lack of professional support services for open-source software remains the stumbling block to its widespread use in business and industry. Business managers want to be able to call a service center when problems arise. In the past, those experiencing problems with open-source applications could send out an e-mail and usually within hours receive the right solution from the developers themselves. These informal networks of technical support can provide the highest possible levels of support, but they are not always immediately available, nor can they scale to meet the growing demands.

  The issue of technical support was important to WDN and GeoCommunity because they knew they would occasionally need a technical safety net. They purposely chose applications with strong development communities in order to get the help they need directly from their web sites. Their technical staff concedes that companies without their own in-house technical staff need more comprehensive support. It's the one issue that continues to scare managers away from open source.

  Fortunately, entrepreneurs always rush to fill a vacuum. Red Hat was one of the first companies to provide support for the Linux operating system, and now a slew of other companies are springing up to provide support, training and consulting services for some of the best open source database applications. The open-source support gap is shrinking fast, which is good news for emerging companies in need of an affordable platform for their growing business.

  In the meantime, the WDN and GeoCommunity remain satisfied with their open-source decision, and the reliability, error reporting, community support, clean designs and strict adherence to industry standards that came with it. It's also refreshing that with open source, there can be no effort to force "lock-in" or add proprietary hooks that will prevent transition to other products in the future. As one senior staff member said, "It's truly been a liberating experience to use good products that were designed simply to meet a need--not to further a corporate agenda."

  ---

  Copyright © 2001, Ned Lilly.
  Copying license http://www.linuxgazette.com/copying.html
  Published in Issue 65 of Linux Gazette, April 2001

  "Linux Gazette...making Linux just a little more fun!"

  ---

  Finding my computer at home from the outside

  By

  ---

  1. Introduction
  2. Perl script uploading ip address.
  3. Webpage and perl script on remote computer.
  4. Cron job I run in the background.
  5. Conclusion
  6. References

  Introduction

  The purpose of this article is to make it so I can find my computer at home when I am traveling around the Bay Area doing computer work, recruiting, and volunteer work. Most of the time, I am busy traveling around, although I am able to work from home half the time now. My computer at home uses a Ricochet modem. The dumb people who promised me a good DSL connection and a satellite connection where I live were a bunch of morons. The max DSL I could get would be 144k (which I found out AFTER I moved in), which is pointless when I already have a Ricochet modem at 128k. Plus, I am facing the wrong way for a satellite connection. Whatever you do, make sure the morons who sell you their apartments have it in the contract that you are promised certain speed connections to the internet, or you can break the contract with no penalty. As soon as it is worth, I am moving. For now, I am stuck with a dial-up connection, which isn't bad most of the time.

  Some people can have static DSL connections, which takes of the problem I have, which is my ip address to the internet changes each time I dial up. I used to email myself the ip address, parse out the data, and put it on a webpage. I have a better solution now. I use ssh to transfer a file to my remote web server once an hour.

  Setting up ssh.

  The version of ssh I am using is 1.2.27. I should be using OpenSSH, but for now, I am using commercial ssh.

  We need to make it so we can transfer files securely from my computer at home to the remote computer. We use the ssh-keygen program (which comes with ssh). Here is a paragraph from the manpage for ssh.

  Ssh implements the RSA authentication protocol automati­ cally. The user creates his/her RSA key pair by running ssh-keygen(1). This stores the private key in .ssh/iden­ tity and the public key in .ssh/identity.pub in the user's home directory. The user should then copy the iden­ tity.pub to .ssh/authorized_keys in his/her home directory on the remote machine (the authorized_keys file corre­ sponds to the conventional .rhosts file, and has one key per line, though the lines can be very long). After this, the user can log in without giving the password. RSA authentication is much more secure than rhosts authentica­ tion.

  So I ran "ssh-keygen" as a user on my computer at home. Then I transferred the ".ssh/identity.pub" file on my computer at home to the remote computer as ".ssh/authorized_keys" for the user "web1" on the remote computer. This makes it so I can login in from home to my remote computer without having to use a password. This can also be used to transfer files.

  rsync -e ssh -av /home/test1/IP.txt web1@somecomputer.com:public_html/IP.txt
  

  Perl script uploading ip address.

  Here is the perl script I use to upload the ip address. You should change values of the usernames and remote computer address. [Text version of this script.]

  #!/usr/bin/perl
  
  use strict;
  
    ### Run ifconfig and store the data in the @Temp list. 
  my @Temp = `/sbin/ifconfig`;
  
    #### Search for ppp
  my $Search = "ppp";
    ### If you are looking for the ip address of your ethernet card, 
    ### uncomment the next line;
  # $Search = "eth0";
  
    ### Make the line we find the ip address blank initially.
  my $Match_Line = "";
  my $Match_Device = "no";
  
    ## Search through the lines, if we find a match, save the lines until
    ## we find a blank line. 
  
  foreach my $Line (@Temp)
    {
      ### If we have a match, abort. 
    if ($Match_Line ne "")   {@Temp = ();}
      ### else, see if we can find a match at the beginning of line;
    elsif ($Line =~ /^$Search/) {$Match_Device = "yes";}
      ### else, if we found the device, and we find the line we are looking for
    elsif (($Match_Device eq "yes") && ($Line =~ /^ +inet/)) 
      {$Match_Line = $Line;}  
    }
  
    ## If our $Match_Line is not blank, split it and get the ip address.
  my $IP = "";
  if ($Match_Line ne "") 
     {
      ### Get rid of stuff before addr:
     my ($Junk,$Good) = split(/addr\:/, $Match_Line,2);
      ### Get rid of stuff after the first space
     my ($Good,$Junk) = split(/ /, $Good,2);
     $IP = $Good;
     }
  
    ## If $IP is not blank, we have something. Save to file and transfer file
    ## to remote site. 
    ### Please don't use the /tmp to store this file, but some other location.
  if ($IP ne "")
    {  
    open(FILE,">/tmp/IP.txt");
    print FILE "$IP\n";
    close FILE;
    system ('rsync -av -e ssh /tmp/IP.txt web1@somecomputer.com:public_html/IP.txt');
    }
     ### Else, we should send ourselves an email, or do something
     ### to let us know it didn't work. This is left as an exercise.
  else {}
  

  Webpage and perl script on remote computer.

  On the remote computer storing the ip address, we need to detect if it is an hour old. If it is less than an hour old, we should print out an error message. So I use this perl script. I name it "/home/web1/public_html/IP.pl". [Text version of this listing.]

  #!/usr/bin/perl
  
  use strict;
  
  print "Content-type: text/html\n\n\n\n";
  
  my $File = "/home/web1/public_html/IP.txt";
  open(FILE,"/home/web1/public_html/IP.txt");
  my $Line = <FILE>;
  chomp $Line;
  close FILE;
  
  my ($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size,
     $atime,$mtime,$ctime,$blksize,$blocks)
       = stat($File);
  my $time = time();
  
  print "<br> Last known ip address was $Line\n";
  print qq(<br> To transfer to the website, 
       <a href="http://$Line">click here</a>\n);
  
  my $Diff = $time - $mtime;
  if ($Diff > 4000) 
    {
    print "<p>ERROR: The ip address should have been updated once an hour, 
    but 4000 seconds has past since the last update.
    <br> $time - $mtime = $Diff \n";
    }
  

  You may want to consider moving this perl script into the normal cgi-bin directory of your web server. Otherwise, here is a dangerous example of how to make it so you can run perl scripts from a user's directory. THIS IS DANGEROUS! If your web server allows any user to execute a perl script, that person can get the web server to do anything they want.

  To make it so you can execute perl scripts on your web server,

  
  <Directory /home/*/public_html>
     ## Options All is reduntant with some of the other options. 
      Options All Indexes FollowSymLinks MultiViews ExecCGI Includes 
      AllowOverride All
      Order allow,deny
      Allow from all
  </Directory>
  
     #### This requires several perl apache modules
   <Files *.pl>
   SetHandler perl-script
   PerlHandler Apache::OutputChain Apache::SSIChain Apache::Registry 
   PerlSendHeader On
   Options ExecCGI
   </Files>
  

  The Cron entry to make it run nightly

  Put this in your crontab on the remote server using the "crontab -e" command.

  #/bin/sh
  
    ### Download every two hours
  1 * * * *   /www/Cron/Remote_Website.pl >> /www/Cron/out  2>&1  
  

  Conclusion

  I know people are probably doing the same thing in different ways. I like this solution because the files are transferred securely. This makes it so people can't see what I am transferring over the internet. So that nobody can get to the file, we should password protect the webpage and perl script that display the ip address.

  References

  1. ssh
  2. OpenSSH
  3. Apache
  4. If this article changes, it will be available here http://www.gnujobs.com/Articles/17/Remote_Website.html

  Mark works as an independent consultant donating time to causes like GNUJobs.com, writing articles, writing free software, and working as a volunteer at eastmont.net.

  ---

  Copyright © 2001, Mark Nielsen.
  Copying license http://www.linuxgazette.com/copying.html
  Published in Issue 65 of Linux Gazette, April 2001

  "Linux Gazette...making Linux just a little more fun!"

  ---

  Learning Perl, part 3

  By

  ---

  
  The trouble with teaching Perl as a first computer language is that your students won't appreciate it till they start learning their second. The trouble with teaching Perl as a second language is that there's no single suitable first language to go in front.
   -- Larry Wall

  When they say that Perl is a `glue language', what they really mean is that it is good for cleaning up after the mistakes of other programs.
   -- Mark-Jason Dominus in comp.lang.perl.misc
   
   

  Overview

  This month, we'll look at Perl's conditional and looping constructs, and look at a few scripts that use them. We will also explore how they work with Perl's variables, and take a quick look at capturing user input. Once you understand this part, I suggest hacking out a couple of experimental scripts and playing with them; sure, you'll make mistakes - but from this point on, you'll actually need to supplement your reading by getting down and dirty. If you don't play, you can't win...
   
   

  Conditionals

  Here are the conditional statements that Perl uses; nothing particularly unusual, if you're used to conditionals in other languages. Perl checks if the condition is true or false, and branches the execution based on that.

  ---

  
  if    ( traffic_light_is_red ) {     # If condition 1 is true, do
             stop;                     # Action 1
  }
  elsif ( traffic_light_is_yellow ) {  # If condition 2 is true, do
        hit_the_gas;                   # Action 2
  }
  else  {
                                       # In all other cases, do
        proceed_with_caution;          # Action 3
  }

  Note that the "elsif" clause isn't required; neither is the "else". Also, note that "else" is the 'catch-all option': if the light is anything except red or yellow - burned out, just got knocked down by accident, etc. - the action is 'proceed_with_caution'.

  Unlike C, even single actions must be enclosed in a block (defined by the curly brackets):

  if ( $tomato eq "red" )   print "Ripe.\n";      # WRONG!
  if ( $tomato eq "red" ) { print "Ripe.\n"; }    # Right

  ---

  unless ( $blarg == $foo ) {          # If condition 1 is false, do
         print "Unequal!.\n";          # Action 1
  }
  else   {                             # Otherwise, do
         print "They're equal.\n";     # Action 2
  }

  Pretty obvious. It may help to think of "unless" as the "if not" conditional. Once again, the "else" is optional. No, there's no such thing as "elseunless". :)
   
   

  Loops

  Ah, wonderful loops. These are the things that make actions happen, as many times as we want, based on a condition. You might even say that this loops are the main reasons for computers in general, their main use as the tool that they are: precise repetitive work. Here are the three most
  common types of loops under Perl:

  ---

  
  while ( $cat eq "away" ) {             # While cond. 1 is true, do
        print "The mice will play.\n";   # Action 1
  }

  ---

  
  until ( $time > 1159 ) {        # While cond. 1 is false, do
       print "It's morning.\n"    # Action 1
  }

  ---

  
  The "for" loop can be implemented in two different ways - one is like the "for" loop in C:

  for ( $n = 99; $n > 0; $n-- ) {
      print "$n bottles of beer on the wall, $n bottles of beer,";
      ...
  }

  In this case, we set $n to an initial value (99), decrement it by 1 each time we go through the loop, and check to make sure that it's greater than 0. If it's not, we exit the loop.

  The second method, somewhat like the Clipper, FoxPro, etc. "foreach" loops, is by far the most common:

  foreach $n ( 0..1000 ) {
          print "Day $n on this deserted island. So far, I've had ";
          print $n * 100, " bananas. I hope I'm rescued soon.\n";
          ...
  }

  It can also be used this way:

  for ( 0..1000 ) {
      print "Day $_ on this deserted island. So far, I've had ";
      print $_ * 100, " bananas. I hope I'm rescued soon.\n";
      ...
  }

  Our old friend, the "$_" (explained in the previous part of this series.) He does indeed come in handy. Note that "foreach" is just an alias for "for", and they can be used interchangeably.

  ---

  All of the above conditionals and loops can also be used as single-statement modifiers, as well:

  print "This is line $_ of 50.\n" for ( 1..50 );

  The above will print 50 lines, numbered in an obvious way.

  print "I've found him!" if /Waldo/;

  The above line will be printed if the default buffer ($_) contains a match for "Waldo".
   

  An interesting fact that combines well with loops and conditionals is that empty variables in Perl return a null value - which is "false". This is perfect for checking them out:

  print if $_;            # Prints $_ if it contains anything

  The next example shows that a zero value is also false:

  print "5280 is true.\n" if 5280;   # This will print.
  print "0 is true.\n" if 0;         # This won't print.
  

  Here's an example with a list:

  while ( @a ) {
        print pop @a;     # "Pop" the last value off @a and print it
        $count =  @a;     # Get the number of elements in @a
        print $count, " elements left in \@a.\n";
  }

  When the last element has been popped off, the loop will end.

  unless ( %hash ) {
         %hash = ( 'first' =>  'Mighty Joe',
                   'last'  =>  'Young',
                   'type'  =>  'gorilla',
                   'from'  =>  'Pangani Mountains',
                   'born'  =>  '1949',
                   'Mom'   =>  'Jill',
                   'Dad'   =>  'Gregg'
         );
  }

  If "%hash" is empty, we populate it with some initial values.
   

  The range operator, which we've used a couple of times so far, is a useful widget: it allows you to specify a range of numbers or letters. Note that the ranges have to be of the same 'kind' - if you specify ('a'..'Z') or ('A'..'z'), the output will not be what you expect. Also, you cannot specify ('z'..'a'); that won't work either. However, there is an easy way to do that:

  foreach $letter ( reverse 'a'..'z' ) {
      print "$letter\n";
  }

  It will also properly increment "letter lists":

  for ( 'aa'..'zz' ) {
      print "$_ ";        # Will print "aa ab ac ... zx zy zz"
  }
   
   

  User Input

  Capturing keyboard input, or input from STDIN in general - such as the lines piped to the input of our script via something like

  cat file | perl_script

   - is easy; it's what Perl's "diamond operator" is for.
   

  while ( <> ) {        # Capture all keyboard or piped input
        print;          # Print each line as long as input exists
  }

  The above works exactly like "cat" - it will print all input piped to it, will "cat" a file if it's run with the filename used as an argument, and will accept (and echo) user input until you hit Ctrl-D or Ctrl-C. It can also be written this way:

  print while <>;

  for a more "Perlish" syntax. Note that "<>" and "<STDIN>" are related but not equivalent:

  print while <STDIN>;

  will respond to keyboard and piped input, but will not print the contents of a file supplied as an argument. I've never found a situation where I needed that kind of functionality, so I simply use "<>".

  If you want to assign user input to a variable, Perl also makes that easy - but there's a bit of a trap built in of which you need to be aware:

  $answer = <>;        # Get the input, assign it to the variable
  if    ( $answer eq "y" ) {
        print "Yes\n";
  }
  elsif ( $answer eq "n" ) {
        print "No\n";
  }
  else {
        print "No idea!\n";
  }

  The above script will always print "No idea!" Hmm... it looks right; what could be the problem?

  The problem is that Perl captures everything that you give it. So, when you type "y", what's the next key you hit? "Enter", that's what! So, the variable stored in $answer is NOT "y", it's "y\n" - the answer and the linefeed. How do we deal with that? Perl, of course, has a function - one you should always use when getting user input:

  chomp ( $answer = <> );

  "chomp" will remove the linefeed, or "end-of-line" character, from the string to which it is applied. It will also remove EOLs from every element of an array which it receives as an argument. The old Perl4 version, "chop", removed the last character from a scalar (or from the elements of the array) no matter what it was; it's still available if you should need it for that purpose, but for taking user input, use "chomp" (also known, via Perl's error messages, as the "safe chop").
   
   

  Exercises For The Mind

  Try building a couple of scripts, just for your own education and entertainment:

  A script that takes a number as input, and prints "Hello!" that many times. As a bonus, check the input for illegal (non-numeric) characters (hint: use //, the match operator.)

  A script that takes the current hour (0-23) as input and says "Good morning", "Dobriy den'", "Guten Abend", or "Buenas noches" as a result. <grin>

  If you come up with something particularly clever, don't hesitate to send it to me for the next part of this series: you'll get the credit for writing it, I'll happily dissect it for you, and we'll both become micro-famous and retire to Belize on the proceeds. <laugh>

  Don't forget: your shebang line should always contain "-w". If you don't ask Perl to help you with your mistakes, you'll be wasting a lot of time. Let the computer do the hard work!
   

  #!/usr/bin/perl -w
  print "See you next month!"
   

  Ben Okopnik
  perl -we'print reverse split//,"rekcah lreP rehtona tsuJ"'
  

  ---

  References:

  Relevant Perl man pages (available on any pro-Perl-y configured
  system):

  perl      - overview              perlfaq   - Perl FAQ
  perltoc   - doc TOC               perldata  - data structures
  perlsyn   - syntax                perlop    - operators/precedence
  perlrun   - execution             perlfunc  - builtin functions
  perltrap  - traps for the unwary  perlstyle - style guide

  "perldoc", "perldoc -q" and "perldoc -f"

  ---

  Copyright © 2001, Ben Okopnik.
  Copying license http://www.linuxgazette.com/copying.html
  Published in Issue 65 of Linux Gazette, April 2001

  "Linux Gazette...making Linux just a little more fun!"

  ---

  So You Like Color !!!
  (The mysterious ^[[ characters)

  By

  ---

  Have you ever redirected the output of a curses program with colors and wondered what those mysterious ^[[ are? Did you ever try to produce colors with a printf without using curses? If the answer to either of these questions is yes, read on...

  This article attempts to explain those mysterious characters that one finds in the output of a curses program which produces colors. Later on, we extend the concept to produce colors with a mere printf.

  Terminal Codes

  In the olden days of teletype terminals, terminals were away from computers and were connected to them through serial cables. The terminals could be configured by sending a series of bytes to each of them. All the capabilities of terminals could be accessed through these series of bytes which are usually called escape sequences because they start with an escape(0x1B) character. Even today with vt100 emulation, we can send escape sequences to the emulator and it will have the same effect on the terminal window. Hence, in order to print color, we merely echo a control code.

  Type this on your console.

          echo "^[[0;31;40mIn Color"
  

  The first character is an escape character, which looks like two characters ^ and [. To be able to print that you have to press CTRL+V and then the ESC key. All the others are normal printable characters. You see the string "In Color" in red. It stays that way and to revert back type this

          echo "^[[0;37;40m"
  

  As you can see it's pretty easy to set color and reset it back. There are a myriad of escape sequences with which you can do a lot of things like moving the cursor, resetting the terminal etc..

  The Color Code:     <ESC>[{attr};{fg};{bg}m

  I'll explain the escape sequence to produce colors. The sequence to be printed or echoed to the terminal is

          <ESC>[{attr};{fg};{bg}m
  

  The first character is ESC which has to be printed by pressing CTRL+V and then ESC on the Linux console or in xterm, konsole, kvt, etc. ("CTRL+V ESC" is also the way to embed an escape character in a document in vim.) Then {attr}, {fg}, {bg} have to be replaced with the correct value to get the corresponding effect. attr is the attribute like blinking or underlined etc.. fg and bg are foreground and background colors respectively. You don't have to put braces around the number. Just writing the number will suffice.

  {attr} is one of following

          0       Reset All Attributes (return to normal mode)
          1       Bright (Usually turns on BOLD)
          2       Dim
          3       Underline
          5       Blink
          7       Reverse
          8       Hidden
  

  {fg} is one of the following

          30      Black
          31      Red
          32      Green
          33      Yellow
          34      Blue
          35      Magenta
          36      Cyan
          37      White
  

  {bg} is one of the following

          40      Black
          41      Red
          42      Green
          43      Yellow
          44      Blue
          45      Magenta
          46      Cyan
          47      White
  

  So to get a blinking line with Blue foreground and Green background, the combination to be used should be

     
  echo "^[[5;34;42mIn color"
  

  which actually is very ugly. :-) Revert back with

  echo "^[0;37;40m"
  

  With printf()

  What if you want to use this functionality in a C program? Simple! Before you printf something print this escape sequence to produce it in the desired color. I have written a small routine textcolor() which does this automatically. You can use it in your programs along with the #define constants. The text version of this program is here

  textcolor()

  #include <stdio.h> #define RESET 0 #define BRIGHT 1 #define DIM 2 #define UNDERLINE 3 #define BLINK 4 #define REVERSE 7 #define HIDDEN 8 #define BLACK 0 #define RED 1 #define GREEN 2 #define YELLOW 3 #define BLUE 4 #define MAGENTA 5 #define CYAN 6 #define WHITE 7 void textcolor(int attr, int fg, int bg); int main() { textcolor(BRIGHT, RED, BLACK); printf("In color\n"); textcolor(RESET, WHITE, BLACK); return 0; } void textcolor(int attr, int fg, int bg) { char command[13]; /* Command is the control command to the terminal */ sprintf(command, "%c[%d;%d;%dm", 0x1B, attr, fg + 30, bg + 40); printf("%s", command); }

  The textcolor() is modeled against the Turbo C API function. You call the function to set the color and then print with a sprintf() (a function used in Turbo C to produce console output in color).

  A Demo of colors

  #include <stdio.h> #define RESET 0 #define BRIGHT 1 #define DIM 2 #define UNDERLINE 3 #define BLINK 4 #define REVERSE 7 #define HIDDEN 8 #define BLACK 0 #define RED 1 #define GREEN 2 #define YELLOW 3 #define BLUE 4 #define MAGENTA 5 #define CYAN 6 #define WHITE 7 #define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0])) char *attrs[] = {"NORMAL", "BRIGHT", "DIM", "UNDERLINE", "BLINK", "REVERSE", "HIDDEN", "EXIT"}; char *colors[] = {"BLACK", "RED", "GREEN", "YELLOW", "BLUE", "MAGENTA", "CYAN", "WHITE", "EXIT"}; void textcolor(int attr, int fg, int bg); int print_menu(char *array[], int n_options, char *title); int main() { int attr, fg, bg; int attr_size, colors_size; attr_size = ARRAY_SIZE(attrs); colors_size = ARRAY_SIZE(colors); while(1) { printf("\n"); attr = print_menu(attrs, attr_size, "Choose the attr you want:"); if(attr == attr_size - 1) break; fg = print_menu(colors, colors_size, "Choose the foreground you want:"); if(attr == colors_size - 1) break; bg = print_menu(colors, colors_size, "Choose the background you want:"); if(attr == colors_size - 1) break; printf("\n"); textcolor(attr, fg, bg); printf("This is what you get if you use the combination %s attribute %s foreground and %s background", attrs[attr], colors[fg], colors[bg]); textcolor(RESET, WHITE, BLACK); system("clear"); } return 0; } int print_menu(char *array[], int n_options, char *title) { int choice, i; for(i = 0;i < n_options; ++i) printf("%d.%s\n", i, array[i]); printf("%s", title); scanf("%d", &choice); return choice; } void textcolor(int attr, int fg, int bg) { char command[13]; /* Command is the control command to the terminal */ sprintf(command, "%c[%d;%d;%dm", 0x1B, attr, fg + 30, bg + 40); printf("%s", command); }

  This program asks the user to play with attributes and colors and shows a string in that color. I usually use it to find out the best combination of colors for my GUIs. Text version of above program is here .

  The Catch

  Then what's the catch? If producing color is so easy, why do people waste their time writing huge programs in curses, which in turn query terminfo in a complex way? As we know, there are many terminals with very few capabilities and terminals which don't recognize these escape codes or need different codes to achieve the same effect. So if you want a portable program which would run on any terminal with the same (or reduced) functionality, you should use curses. Curses uses terminfo to find the correct codes to accomplish the task in style. Terminfo is a big database which contains information about the various functionalities of different terminals.

  But if you just want to write a simple program which produces color on a Linux console or xterm window, you can just use the escape sequences above to do it easily. The Linux console mostly emulates vt100, so it recognizes these escape sequences.

  With tput

  But there is a way to query the terminfo database and do the work. tput is the command which queries the database and executes the functionality you specify. The two capabilities setf and setb are useful to set foreground and background colors. Use this to set foreground color to red and background color to green.

          tput setf 4     # tput setf {fg color number}
          tput setb 2     # tput setb {bg color number}
  

  This can be used in shell scripts where you want. See the tput manual page for additional capabilities of tput. The terminfo manpages contain a lot of information regarding terminal capabilities - how to get and set their values and more. There are two terminfo manpages. "man 5 terminfo" describes the terminfo database. "man 3ncurses terminfo" describes the C functions that use the database.

  These are the color numbers to be passed as arguments to "tput setf" and "tput setb".

          0       Black
          1       Red
          2       Green
          3       Yellow
          4       Blue
          5       Magenta
          6       Cyan
          7       White
  

  Have fun !!!

  References

  • The Text-Terminal-HOWTO
  • Man pages for tput and terminfo.

  ---

  Copyright © 2001, Pradeep Padala.
  Copying license http://www.linuxgazette.com/copying.html
  Published in Issue 65 of Linux Gazette, April 2001

  "Linux Gazette...making Linux just a little more fun!"

  ---

  Book Review: Networking Printing

  By

  ---

  Network Printing
  O'Reilly and Associates
  October 2000
  ISBN 0-596-00038-3
  $34.95

  There are few applications so beneficial, pervasive, and--oftentimes--complex as network printing. Network printing is beneficial because it reduces the number of printers required for an organization. Allowing users to print to a limited set of printers, rather than requiring a printer for each user realizes an obvious reduction in capital cost. This also equates to a savings in space requirements and power consumption. (These are two often overlooked but important factors.) The reason network printing is so pervasive is directly a result of the benefit of network printing--it reduces cost, both in terms of capital outlays and maintenance.

  Unfortunately, network printing can also be quite complex. This is especially true for heterogeneous networks. In a heterogeneous network, not only do administrators need to worry about printers and print servers speaking the same lingo, but also that each device is actually using the same network layer protocols (i.e., TCP/IP). Even when a network is homogenous there can be difficulties, especially in large organizations where printers number in the hundreds or thousands.

  In order to combat this complexity, and with it's resulting rise in cost and overhead, an administrator needs a solid set of documentation and a framework from which to grow. O'Reilly has attempted to satisfy just this need with their release of "Network Printing," by Todd Radermacher and Matthew Gast. "Network Printing," published in October of 2000, provides a step-by-step guide for building an infrastructure to support network printing in heterogeneous networks (and, by extension, homogenous ones as well).

  So what exactly does Radermacher and Gast, the authors, bring to the table? Both Radermacher and Gast have several years of experience in the computer industry. They also both have a very readable writing style, and consistently speak to the reader in the first person. (This helps to engage the reader in the material, and often leads to more readable technical literature.) Now, on to the book!

  In Chapter 1, "A Brief History of Printing and Publishing", Network Printing begins with a general introduction to printing in general. By "in general" I mean the entire field of printing, and not just network printing. The authors give a quick overview of the history of printing, including the introduction of such notables as papyrus scrolls and the Linotype. Personally, I feel this type of material is usually best left to the history books, but you may disagree.

  The second chapter, "Printer Languages," progresses to the more relevant topic of page-description language. A page-description language is the lingua franca used by a print server and a printer. Common examples, and ones that are covered in the book, are Adobe's PostScript and Hewlett-Packard's Print Command Language (PCL). All in all, the authors do a good job of summarizing these languages. However, if you are looking for in-depth coverage, you will need to go elsewhere.

  Chapters 3, 4, and 5 concern three popular UNIX print systems currently in use: BSD, SysV, and LPRng. The emphasis of the book is on using UNIX as the central print server platform for an organization, so the concentration on these systems is important. (However, I would have liked to see more focus on NT print servers.) Special attention is paid to print filters, which form the core of the UNIX print process.

  In Part II, "Front-End Interfaces to UNIX queues," the authors begin with the requisite chapter on Samba. Chapter 6, "Connecting Windows to UNIX Servers: Let's Samba", describes deploying Samba on UNIX machines so that the servers can interface with Windows networks. Certainly, this book is not the end-all for documentation relating to Samba and it's various configuration options, but Gast and Radermacher cover it in enough detail to get the reader up and running.

  After the coverage on Samba and Windows environments, the authors move to a more underserved support issue in many books stressing UNIX solutions: integration with Macintosh and NetWare networks. In Chapters 7 and 8 the authors cover netatalk and ncpfs, respectively. Similar to the Samba chapter, the authors' main focus here is to educate the reader about the aspects of the support software relating to printing.

  In Part III, "Administration," Radermacher and Gast enter into one the more crucial aspects of network printing--effectively and efficiently administering the system. At this point the authors assume you have the knowledge to implement network printing for the various networks covered, and they move to making the system not only effective but also efficient.

  In Chapter 9, "Using SNMP to Manager Networked Printers," the authors demonstrate how to use SNMP to monitor and control your printer infrastructure. Of note is their good overview of SNMP, and review of SNMP agents, such as MRTG. Not the strongest chapter in the book, but more than sufficient.

  Next, in Chapter 10, "Using Boot Servers for Basic Printer Configuration," and Chapter 11, "Centralized Configuration with LDAP," the emphasis is on methods for maintaining a centralized configuration for all of the network printers. In small to medium networks these chapters may not be truly useful, but for large installations, centralized configuration is vital. The chapter on LDAP is especially informative, and offers several insights.

  Finally, in Chapter 12, "Accounting, Security, and Performance," the authors tie many loose ends left from earlier chapters. The main point of this chapter is demonstrating the use of scripts for accounting and monitoring and tuning server performance. The section on security is rather small unfortunately, and I would have liked to see more detail. Alas, it was not forthcoming.

  In conclusion, I think this was a rather well done book. The authors did an excellent job of keeping a rather boring subject (for most of us at least) somewhat upbeat. I also was quite happy to see several rather keen insights, especially the use of LDAP to pull configurations to print servers. If you are a network administrator that is not afraid of Linux or UNIX and need to better organize and control your printer infrastructure then this is an excellent resource.

  ---

  Copyright © 2001, Dustin Puryear.
  Copying license http://www.linuxgazette.com/copying.html
  Published in Issue 65 of Linux Gazette, April 2001

  "Linux Gazette...making Linux just a little more fun!"

  ---

  OO Thinking            By

  ---

  Article #5 - Apr 2001

  Abstract

  
      For those who haven't read the previous articles, be sure to read the statement of purpose first.  This month, we're going to discuss OO thinking.  For those looking to read the whole series locally or information about upcoming articles, you can check the MST page.  For those looking for further information on learning Squeak, here are some good resources.
      This is the last planned article for this series.  The reader interest has been high enough for me to continue with the next series, but unfortunately my available writing time has quickly dwindled :-( as my wife nears her due date :-)  So this will be the last regular article at least for a while.

  Quote of the day

  Reason never changed a man's opinion which by reason he never acquired.
          -- Mark Twain

  OO Thinking

      If you're just getting into OO from another programming background, you'll soon realize that it requires a change in the way that you think, the way you approach problems, and (IMHO) how much fun you're having.  This month, we go over some things to keep in mind when doing OO programming.
   

  Breaking Linear Thinking

      This is the first hurdle I've seen many people trip over.  They're so used to programs with a main() routine of some sort, that when they first dip their toes into the Smalltalk pool they're frightened off by not being able to find a linear beginning, middle, and end of something.  Realize that Smalltalk is about working with a group of collaborating objects.  To be sure, you will need some entry point to your code/application, however it likely be in the form of opening your starting window, then saving/stripping your image.
      Thinking of problems in terms of nouns and verbs (objects and responsibilities) is a more natural way of thinking, and often leads to a much different decomposition of the problem than functional decomposition.  Try to identify which objects are inherit to the problem, which objects need to involved to help out, then think of the most basic responsibilities and distribute them appropriately across the objects.
      This leads us to our next item:  OO programming lends itself well to iterative development.  It's a natural activity to define the basic objects, then start adding basic relationships and responsibilities.  If you find something doesn't fit right, then shift the responsibility elsewhere.  Flesh out your objects and responsibilities over time.
      Try to use short methods to help maximize reuse and maintainabilty.  If you find yourself writing 100 line methods, then you're still thinking linearly.  The average method length varies depending on whom you ask, but it should be short - somewhere around 8 statements or so.  Of course, there's always exceptions to any rule - this is just a rule of thumb.

  Decision Making vs Commanding

      This is what I often think is the most fundamental difference between OO programming and procedural programming.  In procedural programming it's common to do things in terms of decision making.  You do things like:
  • if this, then that, else that
  • • For example, if data is an integer and user input is a float, then convert the float to an integer to add
  • for i = 1 to i = maxRange do this unless i > maxBounds, and if early break condition is met then break out of loop
  • 1 + 2 * 3 = ?
  • • This example uses operator precedence, which is something that most languages have.  The statement is evaluated as: (1 + (2 * 3)) = 7.  But to determine precedence the language needs to make a decision which operator to precede.
      A common problem that arises from decision making programming is that you have similar decision making being done in several parts of a program.  Then when requirements or needs inevitably change, there are many different spots that you need to update/modify your program to update all the decision making spots.

      In OO programming, it's more common to do things in terms of commanding.  You command (or ask if you're polite) objects to do things.  If the object shouldn't do something, or should do something differently, then it should know that.  Since you ask different objects the same thing, and they respond as each of them should, there's no decision making.  You do things like:

  • object doSomething
  • • For example, it doesn't matter that you're adding a float to an int, the float object knows how to add floats to itself, how to add ints to itself, how to add fractions to itself.
  • aCollection do: [:eachElement | eachElement doSomething]
  • • Notice how there are no bounds checking - a collection object already knows how to do that and does it for you.
  • 1 + 2 * 3 = ?
  • • In this example, remember integers are objects too in Smalltalk (part of the pure OO nature of Smalltalk).  So we're asking the object 1 to add itself to 2, then the resulting object to multiply itself by 3.  Hence, the statement is evaluated as: ((1 + 2) * 3) = 9.
    • As a side note, it's funny how often I've seen some of my C++ or Java coworkers flee from Smalltalk because this doesn't make sense to them.  They still haven't entirely made the shift to OO thinking.

  Don't Sweat the Details

      I once heard Alan Knight remark that you know somebody is starting to get Smalltalk if they answer the question:  "How does Transcript show: 'HELLO WORLD' work?" with: "I don't care".   A common theme among Smalltalk newbies is a need to know exactly how everything works, and step through all the methods of the objects from the library that they use.  This is related to linear thinking, in that you need to understand how a linear path flows to determine how it broke down the road.  If you find yourself sweating the details of the class library, then you're probably still in linear thinking mode.
      A related theme is that Smalltalk lends itself to top-down coding.  Put off work as long as possible and put off decisions as long as possible - abstract and stub out reponsibilities if you can.  It's a powerful feeling to define even a trivial system that works, then keep it working as you add real meat to it.  You're most often in a state of things working.

  Simplification by Encapsulation

      Try and group data together with appropriate operations in an object.  If you're acting directly on an object's data in some manner, then you're breaking encapsulation.  If you're doing something like:  anObject aDataAttribute aPartOfAttribute doSomething, then you're breaking encapsulation.
      A nice example of encapsulation is the looping noted above.  The collection class knows how many elements it has, and how to loop over its elements, and you're not concerned with bounds checking nor should you be.

  Reuse

  Opportunities for reuse abound, and not just from the usual place of inheritence

  ...through the class library

      Before coding something, browse the class library to see if it's already been done for you.  Reinventing the wheel is definately non-OO and wastes time.
      Another rule of thumb for knowing when you're getting Smalltalk is the proportion of time you spend browsing the class library to the proportion of time you spend coding.  As you gain experience and familiarity with the library, your proportion of time will go down, but for a beginner you should expect to spend the majority of your time browsing the library and the minority of your time coding.
      An appropriate remark I once heard (sorry, don't remember the source), during a LOC metrics flame war is that Smalltalkers should be measured by the LOC they don't write, as they're saving time and maintentance costs by reusing the class library.

  ...through goodies

      Smalltalk has a rich history and a great user community.  There may be a freeware or opensource goodie out there that will satisfy your needs.  Have a look at the UIUC repository, or search the web or ask the newsgroups for goodies.

  ...by approrpiate responsibilities

      If it isn't your reponsibility, then don't do it (or redo it).  Conversely, avoid responsibilities as much as possible (only take the appropriate responsibilities).  By trying to stick to only appropriate responsibilities, then you're more likely to reuse responsibilities elsewhere in the system.
      For example, don't have the responsibility to login to your application in your client's login GUI (a bad practice in general), if you later have a web GUI, then you need to either copy the login logic to your web GUI, or factor out the login code to a reusable object.

  ...through inheritence

      Now we finally get to reuse through inheritence.  I leave this for last, as reuse through inheritence has been (IMHO) overhyped and often overabused with needlessly deep class hierarchies that complicate maintenance.
      For example, if you're writing a hospital system you'd probably want to reuse a Person's characteristics of firstName, lastName, and socialSecurityNumber by making subclasses of Doctor and Patient.

  Distributing responsibilities

      Watch out for bloated parts of system - you can see this if you're drawing your system out and your diagram looks like an octopus.  This is a sign that there are too many responsibilities on one object, and that object is going to get harder to maintain as it bloats.  You should try and have groups of peer objects collaborating.
      Another warning sign is using a 'manager' object.  Again, there are perfectly good times and uses for a manager object, and it can be difficult to determine if you're abusing a manager object.  I like to use a rule of thumb I heard from Alan Knight: object managers should be like real world managers:  they should not do any real work - they should facilitate or manage interactions between other objects.

  A Sweet Squeak

  This month's sweet squeak is the release of Squeak 3.0! :-)  To be generic as possible, this description covers the scenario where you want to run Squeak in Windoze or Linux.   For this simple path install, on Linux you will need root priviledges.  (Note: you can install without root privilidges if you're familiar with updating your paths, I'm not going to cover that topic in this simple guide)
   

  Step 1:  Downloading Squeak 3.0

  Go to the FTP site: ftp://st.cs.uiuc.edu/pub/Smalltalk/Squeak/3.0 and download:
  • Squeak3.0-win.zip, includes:
  • • Squeak.exe, the virtual machine (only good for Windoze, we'll need to compile a VM for linux)
    • Squeak3.0.image, (can use this on linux or Windoze)
    • SqueakV3.sources, (can use this on linux or Windoze)
  • Squeak-3.0pre2.tar.gz
  • • Source files for compiling the linux VM

  Step 2: Set a base directory to run squeak from

  Assumes your Windoze mount point is /windoze, change for your system.

  Note:  if you don't have/want to run dual boot, just change your install location to be whatever you desire, for example: ~myuserid/squeak3, and delete the unnecessary files: NPSqueak.dll, Squeak.exe, SqueakFFIPrims.dll.

  • Make a /windoze/squeak3 directory
  • Unzip the Squeak3.1-win.zip file into a /windoze/squeak3 directory.

  Step 3: Installing VM for linux

  This is a very easy thing to do - even if you've never programmed or compiled anything in your life before.  Here are the steps:
  1. Unzip Squeak-3.0pre2.tar.gz to wherever (be sure to unzip with directories, this unzips into a Squeak-3.0 directory)
  2. cd to where you unzipped the sources.  (The BUILD.UnixSqueak is a quick-n-easy guide from which these steps were condensed from)
  3. mkdir build
  4. cd build
  5. ../src/unix/configure --bindir="/windoze/squeak3"
  6. make
  7. make install  (NOTE:  here is where you'll need root privilidges with the default install, as stuff is copied to /usr/lib, /usr/man, etc)
     1. Here, you're going to get a couple of errors (unless you're installing to a Linux location), as you can't make links on a Windoze file system
     2. Copy the referenced files to your /windoze/squeak3 directory:
     3. cp /usr/lib/squeak/3.0/squeak /windoze/squeak3
     4. cp /usr/lib/squeak/3.0/inisqueak /windoze/squeak3

  Step 4: Start Squeak :-)

  • cd /windoze/squeak3
  • squeak Squeak3.0final.image
  ...I'll leave starting up Squeak in Windoze as an exercise for the reader  ;-)

  Quick tour

      When I started up the image for the first time, I was pleasently surprised that the default GUI to come up is the newer morphic GUI (as opposed to the older MVC GUI that was mentioned in Article 1).  For the read-along folks, you'll see (click on the below half size images for full size images):

  The entry screen.  The Squeak logo in the top right is an xeyes type of app, where the eyes follow the mouse.
      

  If you put the mouse curor over the logo, you'll notice the pop-up balloon help is enabled.
      

  If you click on the project at the bottom right of the screen, it'll zoom to full screen size as you enter it.
      

  And finally, lets click on the music project to have a look.
      
   

  Looking forward

      Alas, there will be no immediate looking forward due to my time constraints.   The next series I was planning covers some basic programming basics like:  unit testing (SUnit), source code management (change sets and SCAN), an object tour of commonly used objects, control structures, and Squeaklets.

      In the meantime though, I highly recommend downloading v3.0 of Squeak (noted below) and trying out the STP goodies as your first goodie exploration.  They're available from:  http://www.create.ucsb.edu/squeak/STP12.html

      I've enjoyed learning about Squeak over the past few months, and I hope you've enjoyed the series.

  ---

  Smalltalk Code

  Somebody pointed out to me that the ScopedBrowser used in Article 4 doesn't work properly in Squeak v3.0, so here's an updated version.
  Note:  I noticed that SUnit is now included as part of the base image now, so I've included some programmatic unit tests.  After loading the code, if you wish to run the unit tests, do: TestModel openAsMorph, then click the Run button.  You'll notice 8 windows pop up and close, and there shouldn't be any errors listed in the error pane.

  ---

  Copyright © 2001, Jason Steffler.
  Copying license http://www.linuxgazette.com/copying.html
  Published in Issue 65 of Linux Gazette, April 2001

  "Linux Gazette...making Linux just a little more fun!"

  ---

  A Private Home Network

  By

  ---

  1. Introduction

  Until recently I paid little attention to the security of my home network for the following reasons:
  • A dial-up home installation will not attract much attention from crackers.
  • Linux is safe anyhow, compared to MS Windows.
  • The people who put together my Linux distribution have surely taken security into account.
  • I took some security measures (hosts.allow, hosts.deny, ipchains) following the examples in the HOWTO's.
  • I don't understand this security business anyway.
  Common human psychology suggests that this is how many ordinary Linux users think. In my case, unfortunately, all these points turned out to be pure wishful thinking, apart from the last one. Because of  the last one.

  How did I find this out? In order to prepare for the happy day in the future when permanent, high-speed connections to the Internet will be offered in my area, I decided it was a good idea to start investigating security issues. The results were shocking.

  The first shock came from looking at my long-neglected /var/log/syslog* files. A few 'refused connect from' entries. One 'connect from' to ftp which apparently succeeded. Oops. Dial-up Internet users are not overlooked by the crackers after all. And my security is not bullet-proof. Better to spend some time really looking at security. And to try to understand something of it this time. So this meant reading books, FAQ's, HOWTO's, and a lot of articles on the Web; and doing some experiments.

  This is the result of my investigations. Mind: I am not an expert, but just an amateur, a home user trying to make things work. Nothing of this comes with any guarantee.

  2. The system

  I have a very simple home network with two machines:
  • earth is a Win 95 machine without printer or modem.
  • heaven runs Debian Linux 2.1 (with various upgrades). It runs exim (for local mail service and for sending mail to the outside world), qpopper (pop3 server for use by earth), and samba (to provide file sharing and printing to earth). heaven connects on-demand to the ISP, opening a modem (ppp) link. Mail from the outside world is collected by means of fetchmail.
  This local network uses one of the IP address ranges reserved for private networks, 192.168.1.0/24. heaven is 192.168.1.1, earth is 192.168.1.2.

  The contents of /etc/hosts on heaven, and c:\windows\hosts on earth, is:

  127.0.0.1               localhost
  192.168.1.1             heaven.my.home  heaven
  192.168.1.2             earth.my.home   earth

  This shows that my network uses the domain name my.home. This name is unregistered and meant only for local use. Mail to the outside will have its 'message from' and 'envelope from' addresses translated (in the July and September issues, 1999, of LG I described how to do this).

  3. 'Private' home networks

  My notion of security is to have a private network. By this I mean a network that provides no public functions. It does not serve WWW pages or files. You cannot telnet into it. It does not even listen to anything coming in from the outside. If anyone comes knocking, there is simply no response. This idea was recently put forward by Sander Plomp, whose articles at rootprompt.org provided much of the inspiration for this piece.

  A LAN which is not connected to the Internet is a private network by definition. Unplug your modem, and you make your network private. But that is a not the kind of private network that I mean. I want to use the Net, send and receive mail, browse the Web, download files, etc. I just do not want anyone from the outside to enter my network.

  Linux systems generally aren't private networks. By default, the installation procedure of a Linux distribution sets up all sort of nice network services (like telnet, ftp, finger, etc.) which are accessible by anyone in the world, protected (if at all) only by a password. Also, Microsoft Windows home LAN's are generally not private. Connect two Win95 computers and enable 'file sharing', and the whole world can share your files while your Internet connection is up.

  To make non-private networks safe, various techniques are used; passwords of course, and also other security techniques that have been discussed in the Linux Gazette many times, like tcpd (alias tcp wrappers) and kernel-level packet filtering (with ipchains as the user interface). These techniques give some privacy to a system which is essentially public. They are like guards at the door, put there to keep out unwanted characters, while letting the desirable customers in. But why should there be doors at all? I don't want any customers. My network is private!

  If we have servers running, reachable by the outside world, we always have to worry about having made some configuration mistake which can be exploited. Also, server programs often have bugs in them which offer openings to crackers. Only recently one was discovered in named. OK, it was later patched, so people who got the newest version of named do not have to worry anymore about this bug. But what about the next one? Better not to to have any doors at all!

  If you really want to enable services (mp3 distribution, or whatever) for use by computers outside your home, you must study more advanced security techniques. But if you simply want a private network for the home, read on.

  4. How safe is your network?

  To test the safety of a home network, you can have it 'scanned' from the outside, for instance by Secure Design. Click 'scan me now' and 'basic scan'. Try it both from the gateway machine and from the other machines on your network. When I did this, I got the second shock. It was embarrassing. A long list showing possible entry points into my system: Samba shares, telnet, the print service, X, the mail server, ftp, finger, etc. I had some rudimentary safety measures in place, so the system was somewhat protected from serious intrusion (I hope). But I consider it a breach of my privacy that the outside world even knows that I have a mail server (let alone letting them break into it). These services were meant for use within my home network. They are not the business of the outside world in any way.

  So: have your system scanned! Apart from Secure Design, other scan services exist, e.g. Shields Up!, DSL Reports, Sygate Online Services, and many others. A whole lot of them can be found at an Austrian site, Sicherheit im Kabelnetzwerk ('Security in the Cable Network'; there is also an English version with almost the same information). Use several scan services. Print the results. What this 'scanning' actually means will hopefully become clearer in the course of this article.

  You can also 'scan' your system yourself by calling (after su-ing to root) netstat -pan --inet. Use a wide xterm window when doing this, because the output consists of rather long lines. Programs which have 0.0.0.0 in the 'Local Address' column are visible to the whole world!

  5. Servers and clients

  The distinction between servers and clients is not always clear to users. If you want to use ftp, for instance (getting files from, and putting files into, another computer) you use an ftp client program to connect to the other computer. If that is all you want to do with ftp, the client program is all you need. An ftp server is only needed if you want to allow others to get files from, or put them into, your computer. Similarly with telnet: a client program for your own use, a server program for other peoples' use. The client and server programs are completely different and have different names; for instance /usr/bin/telnet for the telnet client, /usr/sbin/in.telnetd for the server. For novices, the distinction is not always clear. If a Linux setup program asks 'shall I set up the ftp server?', users may think 'well eh.. yes, I certainly want to use ftp, so go ahead'. Often you are not even asked, and an ftp server is installed by default.

  One way of creating a private network is not to install servers at all, just clients. But that is too simple if you have a network at home connecting two or more computers. Inside your network you want to telnet from one machine to another, you want to run an internal mail service, etc. In other words you can't do without servers.

  What servers do is listen. They listen for a signal that says: I want your service. The signal (at least for the TCP-based services) is a special IP packet, called a SYN packet, that enters your computer and specifies the number of a service. For instance, the number of the telnet service (that the in.telnetd program, if it is running, listens to) is 23. These numbers are usually called 'port numbers'. If the in.telnetd program is not running, no one listens to SYN packets with the number 23. So, as they say, 'port 23 is closed'.

  Ports do not exist by themselves, like little doors in your computer that you can open or close. A port is open if a server listens to it. Otherwise it is closed. A TCP port comes into existence if there is a program which listens to it, and if not, it does not exist!

  How do the SYN packets get into your computer? In the case of heaven, a packet can get 'into' it in three different ways:

  • packets sent from earth come in via the Ethernet card (also called the eth0 interface); they are addressed to the fixed IP address 192.168.1.1.
  • packets from the outside world come in through the ppp link, or ppp0 interface; that also has an IP address, but is is not fixed. At every Internet session the ISP hands out a 'dynamic' address, valid for this session only.
  • packets can also be sent from heaven itself, addressed to itself as it were. This way of sending packets is often used for testing; the packets are addressed to the so-called loopback interface with address 127.0.0.1. The name localhost refers to this loopback interface, while the name heaven refers to 192.168.1.1. (This is a rather important point: names and IP addresses refer to interfaces, not to computers, although in daily usage this distinction is often blurred.)
  Now the key point is that servers normally listen for any packets with 'their' port number, no matter which way they enter the system. If we want to make a private network, offering no services to the outside world, we must somehow change this.

  It would be nice if all the server programs available on Linux systems had options specifying which interfaces they will listen to. In that case you could just tell all your servers never to listen to the ppp line, and you'd be all set. Hardly any security measures would be needed at all (tcpd, firewalls, etc.); you would only use them 'for good measure', as an extra precaution. Maybe this will happen at some time in the future, but at the moment only a few server programs have this (including the important cases of exim and samba). So we have to do several things to make our network private:

  1. it never hurts to follow the commonly-heard advice to 'close unneeded ports', in other words not to run servers that you do not need.
  2. for services that have the option, make them listen to internal interfaces (eth0 and, if necessary, loopback) only.
  3. the 'super-server' inetd (which is used for 'waking up' a lot of different servers on a Linux system) should be replaced by xinetd, which has the option to listen to internal interfaces only. NOTE: apparently, Red Hat Linux 7.0 installs xinetd by default.
  4. unsafe servers which cannot run from xinetd (like the print server lpd) should, where possible, be replaced.
  5. for the remaining difficult cases you need a firewall that blocks SYN packets from the outside. This could also be used for blocking unwanted UDP and ICMP access.
  6. for 'advanced' security, a few other possibilities suggest themselves. One is not to use IP masquerading and forwarding on your network.

  6. Removing unneeded services

  6.1 Unnecessary inetd services

  First, there are some services which run 'from inetd.conf'. Almost all Linux systems have a 'super-server' called inetd, whose job it is to listen to a lot of ports at the same time, and to 'wake up' services when needed. However, it will also wake up services which you do not need.

  Examples of unneeded services are:

  • the ftp server. I do not plan to serve any files to the outside world, and internally on my network I can use Samba (and smbfs) to transfer files. Not having an ftp server in no way stops you from running an ftp client when you want to, and to exchange files with the outside world. But most distributions, including Debian, install an ftp server by default.
  • the portmapper and anything related to RPC calls (same reason). The portmapper is used for allowing remote procedure calls. Basically you need this if you use NFS. But I do not use NFS (Samba, which I need anyway because there is a Win95 machine on the network, provides enough file sharing facilities). So anything related to the portmapper and RPC can be commented out from inetd.conf.
  • finger and ident. About the usefulness or otherwise of 'ident', opinions seem to be divided. I removed it, and did not suffer any ill effects.
  • several other obviously unnecessary things that can be started up from inetd.conf (like saft).
  • several services in inetd.conf which are used only for testing networks: echo, chargen (pronounced kargen, 'character generator'), discard, daytime, and time. The last two (in case you are worried) have nothing to do with timekeeping on your system; they are just services which will tell your system's time to others. You don't need them, nor any of the other 'test' services.
  These services can all be disabled by commenting out (putting a # character at the beginning of) the corresponding lines in /etc/inetd.conf, and restarting inetd (in Debian: /etc/init.d/inetd restart).

  6.2 Other unnecessary services

  If a service is not woken up by inetd, it runs independently, as a 'daemon' or background program.  Among the daemons which you may not need, apart from the portmapper (if run as a daemon) is a local nameserver (named, pronounced name-dee). There is no reason why a small home network should run such a thing. /etc/hosts and C:\windows\hosts files on your machines, and the addresses of your ISP's name servers in /etc/resolv.conf will enable address lookup on your network.

  There is usually a command to prevent a service from starting automatically upon boot-up, by removing it from the start-up directories; for instance I got rid of a tamagotchi server, automatically installed by Debian 2.1, by calling

  update-rc.d -f /etc/init.d/tama remove

  7. Securing wanted services: non-inetd

  Now we tackle services that we do want to use, but do not want to be visible to the outside world. Often there is some configuration option that will keep the service 'private'. Examples follow.

  7.1 X

  X was designed as a network-oriented window system, but in fact the network features are never used in most setups, and present a security risk. You can eliminate X's network capabilities by starting X with a command-line option: startx -- -nolisten tcp. Secure Design now no longer reports that 'X11 is open'.  To make this permanent, you can make an alias for startx in ~/.bashrc, /etc/profile, or some other good location, like this:

  alias startx="startx  --  -nolisten tcp"

  The -nolisten tcp command should really be in one of the X11 resource files, but so far I haven't found out which one. The 'alias' approach works in any case. To test, run (as root) netstat -pan --inet. X should no longer be mentioned. Of course it would be nicer if we could keep X's network abilities for the local network, only blocking them against outside access, but I couldn't find a way to do that.

  7.2 Samba

  In a Debian system, the configuration file for Samba is /etc/samba/smb.conf (on other systems, it may be /etc/smb.conf). When installing Samba, I chose 'let Samba run as daemons'; it did not work properly from inetd. Any lines referring to netbios (which is what Samba uses) in /etc/inetd.conf must therefore be commented out. Then in /etc/samba/smb.conf, section [global], I added

   bind interfaces only = True
   interfaces = 192.168.1.1

  After /etc/init.d/samba restart, the Samba daemons only listen to our home LAN. They are no longer visible to the outside world. Check with netstat -pan --inet, and by having the system scanned.

  7.3 Exim

  Exim is the mail server (or Mail Transport Agent, MTA) on my system. You may have something else (like sendmail or postfix) but then the same principle applies: your private mail agent should not listen to the outside world. People who send mail to you, and to the other users in your home, send it to mail accounts at the ISP (or to several mail accounts at different ISP's). You retrieve it from there using, e.g., fetchmail. People cannot send mail to your network directly.

  Exim turns out to have an option local_interfaces (which goes into the MAIN CONFIGURATION section of /etc/exim.conf). This is a list of (IP addresses of) interfaces that exim will listen to. This only works when exim runs as a daemon, independent of inetd. To set this up:

  • In /etc/exim.conf, in the MAIN CONFIGURATION section, enter a line:
  • 
    local_interfaces = 192.168.1.1:127.0.0.1
    (apart from the local net, also loopback must be specified, otherwise fetchmail won't work; or you must call fetchmail -S yourmachinename).
  • Comment out the smtp line in /etc/inetd.conf .
  • In /etc/init.d/exim comment out the line exit 0 near the beginning, just after the line #usually this is disabled and exim runs from /etc/inetd.conf. This causes exim to run as a daemon after you call /etc/init.d/exim start. Letting exim run as a daemon means that you have to call /etc/init.d/exim restart after every change to exim.conf.
  Letting exim run as a daemon means that it consumes some cycles and some memory all the time. But as a bonus, exim's RETRY CONFIGURATION now works properly as well, which it never did when running under inetd.

  7.4 Junkbuster

  Junkbuster is an http proxy server which you can configure to keep out ads and other unwanted stuff. It works very well. In a Debian system it listens to port 5865, in other systems to port 8000. This is set in the file /etc/junkbuster/config. By default, junkbuster listens to all interfaces (in other words, to the whole world). However, you can set in the config file

       listen-address      192.168.1.1:5865

  and now only machines on our own network can connect to it (including the gateway machine that junkbuster runs on, heaven in this example, provided heaven is entered in the Netscape 'Preferences/Advanced/Proxies' menu, not localhost).

  7.5 Other (non-inetd) services

  The above examples are only examples. If your system runs other services outside inetd, check their documentation for ways to make them private. For instance, it appears that sendmail can be made to listen only to the local network by means of

       0 DaemonPortOptions=Addr=192.168.1.1

  in the sendmail.cf file. I did not try this.

  7.6 Remaining problem cases, like lpd

  lpd remains a problem. It cannot be made to listen to the internal network only. Basically, it should be replaced by something safe. Sander Plomp recommends replacing it by pdq. I've been too lazy to do this yet, but it certainly needs attention in the near future.

  Other problem cases may remain: servers which you need in your own network but which cannot be made private, and for which private alternatives do not exist. I have this problem with cannaserver, a system for inputting Japanese characters from the keyboard. Such services must be screened from the outside world by means of a packet-filtering firewall. See section 10 of this article.

  8. Masking the inetd services through xinetd

  By now the list of visible services on the system, according to the Secure Design scan, has become:
  • telnet
  • pop3
  • lpd (the print system)
  Much fewer than there used to be, but still far too much. Telnet and pop3 are started by inetd/tcpd and thus are secured by hosts.allow and hosts.deny, but I'm not sure that this protection is 100% cracker-proof. lpd remains totally 'open', as far as I can guess, unsecured. It cannot, I think, be started from inetd.

  8.1 Replacing inetd

  Security for a home network really requires replacing inetd by something which can distinguish between requests for service from the local network and from the outside. Plomp recommends tcpserver; I tried xinetd. First kill inetd, then install xinetd. Important: The Debian script /etc/init.d/xinetd not only starts the xinetd daemon by itself, but also the portmapper. We do not need/want the portmapper, which is used for RPC calls and NFS, which we do not use. So anything related to the portmapper in /etc/init.d/xinetd must be commented out (# at the beginning of the line).

  One way to configure xinetd for telnet and pop3 is to put in /etc/xinetd.conf:

  defaults
      {
        instances = 10
        log_type = SYSLOG daemon
        log_on_success += DURATION HOST USERID
        log_on_failure += HOST
        interface = 192.168.1.1
      }

  service telnet
      {
        socket_type = stream
        wait  = no
        user  = root
        server = /usr/sbin/in.telnetd
      }

  service pop-3
      {
        socket_type = stream
        wait  = no
        user  = root
        server = /usr/sbin/in.qpopper
      }

  So apart from a general 'defaults' section which specifies the interface, there is a separate section for each service that you want to run. Although the format is completely different, the data for the various sections can be found in your existing inetd.conf. See also man xinetd.conf.

  I started xinetd and verified that is is now possible to telnet to heaven both from heaven itself and from earth. However, Secure Design no longer reports that my system has open telnet and pop3 ports! Success! NOTE: from my own machine, telnet heaven succeeds, but telnet localhost does not. xinetd can only bind to one interface; in this case 192.168.1.1, not at the same time to localhost, which is the loopback interface (127.0.0.1).

  By now, all other services in /etc/inetd.conf have been commented out. Therefore inetd no longer does anything and we can get rid of it in the boot-up scripts. In Debian, it goes like this:

  update-rc.d -f /etc/init.d/inetd remove

  Its place is taken by xinetd:

  update-rc.d xinetd defaults

  OK; another step towards security successfully taken.

  The output of netstat -pan --inet is now something like:

  heaven:~# netstat -pan --inet
  Active Internet connections (servers and established)
  Proto  Local Address     Foreign Address  State   PID/Program name
  tcp    127.0.0.1:25      0.0.0.0:*        LISTEN  11391/exim
  tcp    192.168.1.1:25    0.0.0.0:*        LISTEN  11391/exim
  tcp    192.168.1.1:139   0.0.0.0:*        LISTEN  10761/smbd
  tcp    192.168.1.1:5865  0.0.0.0:*        LISTEN  1670/junkbuster
  tcp    192.168.1.1:110   0.0.0.0:*        LISTEN  161/xinetd
  tcp    192.168.1.1:23    0.0.0.0:*        LISTEN  161/xinetd
  tcp    0.0.0.0:515       0.0.0.0:*        LISTEN  148/lpd MAIN
  udp    192.168.1.1:138   0.0.0.0:*                10759/nmbd
  udp    192.168.1.1:137   0.0.0.0:*                10759/nmbd
  udp    0.0.0.0:138       0.0.0.0:*                10759/nmbd
  udp    0.0.0.0:137       0.0.0.0:*                10759/nmbd
  raw    0.0.0.0:1         0.0.0.0:*         7      -
  raw    0.0.0.0:6         0.0.0.0:*         7      -

  Almost all services now listen to a local interface. The print system is the exception: it listens to address 0.0.0.0 (i.e., everywhere) on port 515. Sure enough, if the system is scanned now, only port 515 is reported as 'open'. In fact some Windows-oriented scan services will report your system as totally 'closed', because they do not scan port 515.

  9. What about IP Masquerading?

  For ages I have used IP Masquerading, to give the Windows box in my home access to the Internet. Or so I thought. When recently, in the course of my investigations into system safety, I switched off Masquerading, the Windows box could use the Internet as before.

  What happened? Simply that earth, the Windows machine, only does three Internet-related things:

  1. e-mail, which does not require earth to connect to the Internet, only to the smtp and pop3 servers running on heaven;
  2. Web browsing; for this, earth connects to junkbuster on heaven, again not directly to the Internet;
  3. fetching the outside mail from the ISP; for this, earth's user telnets to heaven and runs fetchmail from there.
  So, ever since I installed junkbuster about six months ago, earth has never approached the Internet directly, and Masquerading is now superfluous. I had not realized this. Inadvertently I had created a 'proxying firewall'. This means that Masquerading can - and must - now simply be switched off. This has several advantages:
  • Simplicity: ipchains (see the next section) no longer has a FORWARD chain, so we don't have to worry about it. We do not need to set up DNS on earth (entering nameserver addresses).
  • Security: if ever some Trojan becomes established on earth, it will not be able to contact its evil accomplices through the Internet.
  The only 'downside' is that earth is now restricted to e-mail and http only. No ping and telnet to the outside world, no ftp, Real Audio, chat, etc. But for the moment mail and http is all that's required. If other services become necessary on earth, I suppose I shall have to install proxies for them on heaven.

  To switch IP masquerading and forwarding off, in Debian, you do

  • change the first line in /etc/network/options so it says ip_forward=no
  • disable forwarding in the kernel by means of  echo 0 > /proc/sys/net/ipv4/ip_forward
  • remove any special commands for Masquerading, like ipchains -A forward -s 192.168.1.0/24 -j MASQ from your startup scripts, ip-up script, or wherever you had them.

  10. Closing the last doors

  Let's recapitulate: by eliminating services, reconfiguring other services so they don't listen to the gateway interface, by wrapping others inside xinetd, and by turning off Masquerading, we have created a system which is already quite secure without a firewall. Now it is time to add the final touch: we build a packet-filtering firewall around the system using ipchains. This should be the last step, not the first.

  One often reads the advice to configure ipchains in such a way that 'everything is blocked by default', and then to make exceptions for the things that you want to allow. Theoretically this may be the right thing, but in practice it leads to much frustration. If everything is blocked, your system will basically not work. You are more or less groping in the dark when it comes to deciding what you have to allow. So I allow everything by default, and then add restrictions one by one. If the system breaks (e.g. no ping, or no Web page viewing) the last restriction has been too drastic, and must be undone. Setting the default policy of a chain to DENY or REJECT can then (again) be the last step, not the first.

  I started by taking down the firewall (ipchains -F) and then running a simple firewall script with one rule:

  #!/bin/sh
  # simple firewall

  ipchains -F input
  ipchains -P input ACCEPT
  ipchains -A input -i ppp0 -p TCP --syn -j DENY -l

  This blocks SYN packets coming from the outside interface, enhancing the privacy of the system very considerably. Nobody from the outside can start a connection; outside scan services report that the site is completely closed (some now even call it 'stealthed' or 'invisible'). But we can add more restrictions. That is, we can use more general DENY/REJECT rules, and more specific ACCEPT rules.

  Before you add restrictions, it is useful to do some experiments. You can make ipchains-type rules which let packets through while logging them (-j ACCEPT -l). So even if (like me) you do not really know which packets to block, you can see what is going on 'normally' by keeping a window open with tail -f /var/log/syslog in it. Then afterwards you can make rules to block packets which are not 'normal'. I strongly advise you to do your own experiments, and to make rules based on your own understanding.

  After a few such experiments, my firewall script in /etc/ppp/ip-up.d looks as follows. This assumes you have no nameserver running, but have the addresses of TWO nameservers provided by your ISP in /etc/resolv/conf. Mind the important backquotes (`)! They may disappear if you cut-and-paste from this page.

  #!/bin/sh
  # A slightly more complicated firewall

  # Find external name server addresses
  ns="`grep nameserver /etc/resolv.conf | awk '{print $2}'`"
  nameserver1="`echo $ns | sed -e 's/ .*//'`"
  nameserver2="`echo $ns | sed -e 's/.* //'`"

  # Set up INPUT rules
  ipchains -F input
  ipchains -P input ACCEPT

  # Block outside input from reserved address ranges
  ipchains -A input -i ppp0 -s 10.0.0.0/8      -j DENY
  ipchains -A input -i ppp0 -s 172.16.0.0/12   -j DENY
  ipchains -A input -i ppp0 -s 192.168.0.0/16  -j DENY

  # Block TCP connections from the outside
  ipchains -A input -i ppp0 -p TCP --syn -j DENY -l

  # Block all UDP except nameserver replies
  ipchains -A input -i ppp0 -p UDP -s $nameserver1 53 -j ACCEPT
  ipchains -A input -i ppp0 -p UDP -s $nameserver2 53 -j ACCEPT
  ipchains -A input -i ppp0 -p UDP -j DENY -l

  # Allow (for now) but log all ICMP
  ipchains -A input -i ppp0 -p ICMP -j ACCEPT -l

  # From local net, allow only packets to us and broadcasts
  # Forwarding is off, other packets won't go anywhere, but
  # now we can log them to detect illegal activity on our net
  ipchains -A input -i eth0 -d 192.168.1.1   -j ACCEPT
  ipchains -A input -i eth0 -d 192.168.1.255 -j ACCEPT
  ipchains -A input -i eth0 -j REJECT -l

  # Set up OUTPUT rules
  ipchains -F output
  ipchains -P output ACCEPT

  # Don't send packets out to reserved address ranges
  ipchains -A output -i ppp0 -d 10.0.0.0/8      -j REJECT
  ipchains -A output -i ppp0 -d 172.16.0.0/12   -j REJECT
  ipchains -A output -i ppp0 -d 192.168.0.0/16  -j REJECT

  # Block all UDP except nameserver requests
  ipchains -A output -i ppp0 -p UDP -d $nameserver1 53 -j ACCEPT
  ipchains -A output -i ppp0 -p UDP -d $nameserver2 53 -j ACCEPT
  ipchains -A output -i ppp0 -p UDP -j REJECT -l

  # Allow (for now) ICMP to the outside, but log
  ipchains -A output -i ppp0 -p ICMP -j ACCEPT -l

  # We do not have FORWARD rules; forwarding is off

  Such a firewall (which you should adapt to your personal tastes and needs) will provide an extra 'shell' around the system. But basically, the security of your system should not depend on the firewall; if only because firewalls are complicated things, and it is far too easy to make mistakes with them. Many other things can be done first to ensure the privacy of your network.

  11. References and further reading

  1. Amateur Fortress Building in Linux, part 1, by Sander Plomp (rootprompt.org)
  2. Amateur Fortress Building in Linux, part 2, by Sander Plomp (rootprompt.org)
  3. Real World Linux Security, by Bob Toxen (Prentice-Hall, 2001).
  4. What is the difference between REJECT and DENY? (Linux@home)
  5. Ipchains log format (Linux@home). For understanding what you see while running tail -f /var/log/syslog.
  6. ICMP Type numbers (IANA)
  7. Setting Up Mail for a Home Network Using Exim (Linux Gazette, July 1999)
  8. Experiments with SMTP (Linux Gazette, September 1999)

  ---

  Copyright © 2001, Jan Stumpel.
  Copying license http://www.linuxgazette.com/copying.html
  Published in Issue 65 of Linux Gazette, April 2001

  "Linux Gazette...making Linux just a little more fun!"

  ---

  Speeding Up Your Net Browsing with PDNSD Domain Name Caching

  By

  ---

  1. Where to find this document
  2. About PDNSD
  3. Installation
  4. Sample configuration file
  5. Tweaking configuration files
  6. FAQs
  7. Credits

  1. Where to find this document

  
  http://geocities.com/sunil_tt/pdnsd.txt

  2. About PDNSD.

  DNS is is the Domain Name System. DNS converts machine names to the IP addresses that all machines on the net have. Name serving on Unix is commonly done by a program called named. This is a part of the ``BIND'' package which is coordinated by Paul Vixie for The Internet Software Consortium.

  PDNSD is a caching DNS proxy server. Unlike BIND, it saves the RAM cache to a file and the same is read up by PDNSD for the next Dial-Up session. BIND when acting as a cacheing nameserver on your local Dial-Up machine stores/caches the name to number translation data in your RAM. This is not written back to the hard-disk upon disconnection 'coz it is not intended for a user/site who/which is not always connected to the Net.

  PDNSD can be configured to speed up Net Surfing on a Dial-Up connection. Since DNS resolution is referenced from the cached file, time is not wasted on the name to number lookup. This in turn speeds up the name to number translation, which actually accelerates your surfing.

  PDNSD is distributed under the GNU/GPL and is available for download at: http://home.t-online.de/home/Moestl/

  Redhat RPMS are at:

  http://home.t-online.de/home/Moestl/

  Debian DEBS are at:

  ftp://ftp.debian.org/debian/pool/main/p/pdnsd/pdnsd_1.1.2.a-2_i386.deb

  3. Installation.

  Download pdnsd-<version>.tar.gz from the above source.

  Decompress and untar using

  tar zxvf pdnsd-<version>.tar.gz

  Change directory to pdnsd-<version> and type

  $ ./configure

  Configure script accepts a number of parameters, see manual.txt file supplied with the PDNSD source. Command line parameters --prefix and --with-distribution are interesting.

  $ ./configure --help

  will list all options

  I am assuming that you have not specified any command line options. Makefile generated by configure will have the following defaults: (It is worth taking a look at the generated Makefile)

  Default installion directory for PDNSD is /usr/local (can be changed with --prefix option to configure). Default location of the PDNSD cache is /var/cache/pdnsd. PDNSD configuration file pdnsd.conf will be found in /etc.

  Now type:

  $ make

  This will compile pdnsd. I did not face any problem compiling it on both Debian 2.2 and Redhat 6.1. Next su to root, as installation requires root previleges. Then type:

  # make install

  This step will do the following (quoted from pdnsd manual.txt):
  1. Copies pdnsd to $(prefix)/sbin/

  2. Copies pdnsd-ctl to $(prefix)/sbin/

  3. Copies docs/pdnsd.conf (a sample configuration) to /etc/ (and backs up /etc/pdnsd.conf to /etc/pdnsd.conf.old). If you have an /etc/pdnsd.conf.old you do not want to be overwritten, save it to another place/name before doing 'make install'

  4. Creates your cache directory if it is not there. After installation, you should check the file permissions and edit /etc/pdnsd.conf to fit your needs . If you use the run_as option, please make sure that your cache directory is owned by the user you specified with this option! Please note that the permission issue has been fixed as of the latest releases. Now /usr/local/sbin will contain two binaries pdnsd and pdnsd-ctl. 'pdnsd' is the proxy DNS daemon and 'pdnsd-ctl' is a program to control the cache.

  The cache is located at /var/cache/pdnsd/pdnsd.cache.The cache file size will be 4 initially and will grow as and when you browse.This aspect of the /etc/pdnsd.conf viz. 'perm_cache=<value>;'. By default it is set as 512(KB).Increase it according to your judgement and a safe value would be 2048(KB) for a machine having 64MB RAM. The cache file size will be 4 bytes initially and will grow as and when you browse. Cache growth will be observed only after a reboot or after restart of the PDNSD daemon. This is due to the fact that PDNSD saves the RAM cache upon exit only.

  PDNSD must be started up each time you boot the system . For this, you have to install start up scripts. rc folder of the source distribution contains startup scripts for Redhat, SuSE and Debian. I have not tested the SuSE scripts.

  Do the following depending on your distribution.

  3.a) Debian GNU/Linux.

  Copy pdnsd-{version}/src/rc/Debian/pdnsd to /etc/init.d and type update-rc.d Stop bind if you have it installed on your system. Edit /etc/resolv.conf and add the following.

        nameserver 127.0.0.1
   

  Comment out entries for all other name servers. Start pdnsd by typing /etc/init.d/pdnsd start. Test pdnsd by typing nslookup. On my system it diplays:

       Default Server: debian
       Address: 127.0.0.1
       >
  

  Stop pdnsd by typing /etc/init.d/pdnsd stop.
  Fire up your editor and add a line like this to the end of your /etc/hosts file:

       127.0.0.2    testhost
  

  Save the file and start pdnsd once again. Type nslookup. Inside nslookup type 'testhost'.

      > testhost
        Server: debian
        Address: 127.0.0.1
        Non-authoritative answer:
        Name: testhost
        Address: 127.0.0.2
  

  If this answer is obtained it shows that your pdnsd is working (remember to remove the last line from /etc/hosts)

  3.b) Redhat Linux

  
  Copy pdnsd-{version}/src/rc/Redhat/pdnsd to /etc/rc.d/init.d Stop bind if you have it installed on your system. Edit /etc/resolv.conf and add the following

      nameserver 127.0.0.1
   

  Comment out entries for all other name servers. Start pdnsd by typing /etc/rc.d/init.d/pdnsd start. Test pdnsd by typing nslookup. On my system it diplays

      Default Server: Redhat
      Address: 127.0.0.1
      >
  

  Stop pdnsd by typing /etc/rc.d/init.d/pdnsd stop.
  Fire up your editor and add a line like this to the end of your /etc/hosts file.

   
      127.0.0.2    testhost
  

  Save the file and again start pdnsd. Type nslookup. Inside nslookup type 'testhost'.

    > testhost
    Server: Redhat
    Address: 127.0.0.1
    Non-authoritative answer:
    Name: testhost
    Address: 127.0.0.2
   

  If this answer is obtained it shows that your pdnsd is working (remember to remove the last line from /etc/hosts)

  4. Sample configuration file.

  My pdnsd.conf looks like this

  ---

  global {
   perm_cache=2048;
   cache_dir="/var/cache/pdnsd";
   max_ttl=204800;
   run_as="nobody";
   paranoid=on;
   server_port=53;
   server_ip="127.0.0.1";
  }
  server {
   ip="202.54.6.5";
   timeout=260;
   interval=900;
   uptest=none;
   ping_timeout=500;
   purge_cache=off;
   caching=on;
  }
  server {
   ip="202.54.1.30";
   timeout=260;
   interval=900;
   uptest=none;
   ping_timeout=500;
   purge_cache=off;
   caching=on;
  }
  server {
   ip="202.9.128.6";
   timeout=260;
   interval=900;
   uptest=none;
   ping_timeout=500;
   purge_cache=off;
   caching=on;
  }
  source {
   ttl=86400;
   owner="localhost.";
   serve_aliases=on;
   file="/etc/hosts";
  }
  /*
  rr {
   ttl=86400;
   owner="localhost.";
   name="localhost.";
   a="127.0.0.1";
   soa="localhost.","root.localhost.",42,86400,900,86400,86400;
  }
  rr {
   ttl=86400;
   owner="localhost.";
   name="1.0.0.127.in-addr.arpa.";
   ptr="localhost.";
   soa="localhost.","root.localhost.",42,86400,900,86400,86400;
  } */

  ---

  This is a sample working configuration (DNS servers are of VSNL an Indian ISP).You must edit servers section of pdnsd.conf to suit your needs.(Fill in DNS servers of your ISP aginst IP entry). Start PDNSD once more and connect to the Internet. Type nslookup and do a query for say, yahoo.com. The server will respond somthing like:

  > yahoo.com
  Server: debian
  Address: 127.0.0.1
  Non-authoritative answer:
  Name: yahoo.com
  Addresses: 204.71.200.245

  Stop PDNSD and disconnect from the Internet. Start PDNSD again and query for yahoo.com through nslookup. If you are geting the same answer as above , fine have a coffee and relax. Else if, there is something wrong and ....??

  5. Tweaking configuration files.

  If you are using BIND as your primary nameserver, one can very well make PDNSD the secondary one. But here you have Catch-22 situation, on which local IP and port would you make PDNSD listen ? Look at ragOO's pdnsd.conf file and named.conf file:

  [pdnsd.conf]

  global {
  perm_cache=2048;
  cache_dir="/var/cache/pdnsd";
  max_ttl=604800;
  run_as="nobody";
  paranoid=off;
  server_port=53
  server_ip="127.0.0.2";
  }

  [named.conf--relevant section only]

  options {
  directory "/var/cache/bind";
  forward first;
  forwarders {127.0.0.2;202.54.6.1;202.54.1.30};
  };

  ragOO's GNU/Linux machine has local (lo) IP addresses from 127.0.0.1 to 8.This is the same in all GNU/Linux systems and one has the option to specify 127.0.0.2 to be the alternate local server. PDNSD listens on Port 53 and note that 127.0.0.2 is the first forwarder in named.conf. This means that your machine/BIND looks up the PDNSD cached records for a number match of the address you/the client program has requested, if not there then it queries the DNS resolvers of your ISP; in order it goes.
  
  

  6. FAQs.

  The follwing question and answer is from the correspondence I had with Thoams Meostl author of pdnsd.

  Q. I had some problem with your default installation. The cache was not growing. It was stuck at 4 bytes. I changed permissions to 'nobody' and it started growing. Probably a problem with my configuration. Will you please let me know the correct file permissions for /var/cache/pdnsd and /var/cache/pdnsd/pdnsd.cache ?

  A. The best thing is to give the user who runs pdnsd write permissions to the cache directory (and of course to the cache file): chown <user> /var/cache/pdnsd chmod 0700 /var/cache/pdnsd chown <user> /var/cache/pdnsd.cache chmod 0600 /var/cache/pdnsd/pdnsd.cache

  Where the permissions can of course be more liberal, if you want. The ones given are the minimum required permissions. The default permissions "make install" sets on the files are also OK. The only important thing is to chown the file. Normally, "make install" should also chown the cache file (maybe a bug? If it didn't for you, please drop me a mail).

  7. Credits.

  Thanks to the author of this nifty utility, Thomas Moestl for clarifying certain points and doubts. He made me a better user of PDNSD :-) !

  Thanks to and (ragOO) for mentioning about 'pdnsd' during one of the ILUG-Cochin meets. ragOO edited and modified the draft heavily and encouraged me to keep on modifying the draft.

  Last but not the least, to all users of this elegant program who may have found this rant useful. Enjoy....;

  ---

  Copyright © 2001, Sunil Thomas Thonikuzhiyil.
  Copying license http://www.linuxgazette.com/copying.html
  Published in Issue 65 of Linux Gazette, April 2001

  "Linux Gazette...making Linux just a little more fun!"

  ---

  The Back Page

  • About This Month's Authors
  • Not Linux

  ---

  About This Month's Authors

  ---

  Shane Collinge

  Part computer programmer, part cartoonist, part Mars Bar. At night, he runs around in a pair of colorful tights fighting criminals. During the day... well, he just runs around. He eats when he's hungry and sleeps when he's sleepy.

  Fernando Correa

  Fernando is a computer analyst just about to finish his graduation at Federal University of Rio de Janeiro. Now, he has built with his staff the best Linux portal in Brazil and have further plans to improve services and content for their Internet users.

  Rahul Joshi

  I am a final year computer engineering student from the Government College of Engineering, Pune, India. I have been using Linux for about 2 years. I have also contributed to the Linux Documentation Project by maintaining the Linux Swap Space mini HOWTO. I was introduced to PVM and MPI during my final year project and I have implemented some brute force programs of our project on the PARAM 1000 Supercomputer at the Center for Developement of Advanced Computing, University of Pune, using both the PVM and MPI Libraries.

  Ned Lilly

  Ned Lilly is vice president of hacker relations for Great Bridge, a company formed to promote, market and provide professional support services for PostgreSQL, the open source database, and other open source business solutions. He can be reached at .

  Mark Nielsen

  Mark works at ZING (www.genericbooks.com) and GNUJobs.com. Previously, Mark founded The Computer Underground. Mark works on non-profit and volunteer projects which promote free literature and software. To make a living, he recruits people for GNU related jobs and also provides solutions for web/database problems using Linux, FreeBSD, Apache, Zope, Perl, Python, and PostgreSQL.

  Ben Okopnik

  A cyberjack-of-all-trades, Ben wanders the world in his 38' sailboat, building networks and hacking on hardware and software whenever he runs out of cruising money. He's been playing and working with computers since the Elder Days (anybody remember the Elf II?), and isn't about to stop any time soon.

  Pradeep Padala

  I am a software engineer at Hughes Software Systems. I love hacking and adore Linux. I graduated this year with a B.E (equivalent to B.S) in Computer Science and Engineering. My intersets include solving puzzles and playing board games. I can be reached through or http://pradeeppadala.homestead.com.

  Dustin Puryear

  Dustin is a professional working in the Information Systems industry. He is author of "Integrate Linux Solutions into Your Windows Network," as well as numerous articles for both paper- and online publications. He may be reached at .

  Jason Steffler

  Jason is a Software Architect for McHugh Software International.  His computer related interests include: OO programming & design, Smalltalking, the peopleware aspects of software, and noodl'n around with Linux.

  Jan Stumpel

  Jan lives in Oegstgeest, The Netherlands. He has been a Linux user since 1995. At the moment he is trying to get a Debian installation just right.

  Sunil Thomas Thonikuzhiyil

  I teach Computer Science at College of Applied Sciences Calicut,India. I have been hooked on linux since 1996. I have a Masters in Computer Science from Cochin University. I am interested in all sorts of operating systems. In my free time I love to listen to Indian classical music.

  ---

  Not Linux

  ---

  World of Spam

  Some of the funnier spams found in the Gazette mailbox.

  Why not OWN your very own profit center gazette?

  Design it to advertise the program(s) you want to see placed up front! Tired of advertising for others? gazette, don't settle for a small part of the pie...

  gazette, YOU can have it ALL!! gazette, receive a monthly check for referrals generated from your very own profit center! Allow the system to auto- matically send out a personalized message on your behalf...

  ---

  I have a domain name for sale that will be exceptionally good for your business: www.midwifes.com.

  Your business will be seen as being more successful. There is prestige in owing a Dot Com Domain.

  Your business will be seen as being more creditable. A Dot Com domain gives your business a more established feel.

  [Doesn't he know that midwives is spelled with a "v"? Or is it intentionally misspelled because www.midwives.com is already taken?]

  ---

  [This one isn't spam but it's funny anyway.]

  Indonesia's First Penguin Meets Kangaroos

  Trabas, the first Indonesian Penguin, will meet the Kangaroos at The Linux Business Expo & Open Source Conference on 7-9 March 2001 at Sydney Convention & Exhibition Center, Australia.

  [Trabas makes an Internet Account Management & Billing System (IAMBS) for ISPs and other Internet-based service providers, as well as other software products.]

  ---

  You are receiving this E-mail because you signed up for Big Daddy up dates and you showed interest in motorcycles or motorcycle products. If you are looking for an E-Zine for bikers look no further!

  ---

  Do you have an interest in Affordable Personal Alcohol Detectors??? Please visit our Drunk Driving Prevention Center at...

  ---

  This is not considered SPAM.

  [By whom?]

  ---

  You and I have not met, but because you?re a respected business professional with an interest in improving employee productivity, I would like to offer you a free preview of the Professional Selling SkillMap(tm).

  ---

  I found your address on a site about wine and spirits, cigar and good living. X is a virtual club for all those interested in wine in both a professional and personal capacity. You too can be among our 6992 members to receive our free weekly bulletin.

  [And you too can be among the 6048 who receive an lg-announce message every month.]

  ---

  We are responding to your request for FREE analysis of your site... We feel there is very substantial potential to promote your site on the Internet...

  Please REPLY to this email and include your full name, telephone number and URL.

  [What?! They are responding to us and they're asking for our URL? I thought they already had it....]

  ---

  FIRST, I MUST SOLICIT YOUR CONFIDENCE IN THIS TRANSACTION... YOU HAVE BEEN RECOMMENDED BY AN ASSOCIATE WHO ASSURED ME IN CONFIDENCE OF YOUR ABILITY AND RELIABILITY TO PROSECUTE A TRANSACTION OF GREAT MAGNITUDE INVOLVING A PENDING BUSINESS TRANSACTION REQUIRING MAXIMUM CONFIDENCE.

  WE ARE TOP OFFICIALS OF THE FEDERAL GOVERNMENT CONTRACT REVIEW PANEL WHO ARE INTERESTED IN IMPORTATION OF GOODS INTO OUR COUNTRY WITH FUNDS WHICH ARE PRESENTLY TRAPPED IN [West African country].

  THE SOURCE OF THIS FUND IS AS FOLLOWS : DURING THE REGIME OF OUR LATE HEAD OF STATE, [name], THE GOVERNMENT OFFICIALS SET UP COMPANIES AND AWARDED THEMSELVES CONTRACTS WHICH WERE GROSSLY OVER-INVOICED IN VARIOUS MINISTRIES... WE HAVE IDENTIFIED A LOT OF INFLATED CONTRACT FUNDS WHICH ARE PRESENTLY FLOATING IN THE CENTRAL BANK OF [that country].

  HOWEVER, DUE TO OUR POSITION AS CIVIL SERVANTS AND MEMBERS OF THIS PANEL, WE CANNOT ACQUIRE THIS MONEY IN OUR NAMES. I HAVE THEREFORE, BEEN DELEGATED AS A MATTER OF TRUST BY MY COLLEAGUES OF THE PANEL TO LOOK FOR AN OVERSEAS PARTNER INTO WHOSE ACCOUNT THE SUM OF US$25,000,000.00 (TWENTY FIVE MILLION UNITED STATES DOLLARS) WILL BE PAID BY TELEGRAPHIC TRANSFER. HENCE WE ARE WRITING YOU THIS LETTER.

  WE HAVE AGREED TO SHARE THE MONEY THUS:
  1. 70% FOR US (THE OFFICIALS)
  2. 20% FOR THE FOREIGN PARTNER (YOU)
  3. 20% TO BE USED IN SETTLING TAXATION AND ALL LOCAL AND FOREIGN EXPENSES.

  IT IS FROM THIS 70% THAT WE WISH TO COMMENCE THE IMPORTATION BUSINESS.

  PLEASE NOTE THAT THIS TRANSACTION IS 100% SAFE AND WE HOPE THAT THE FUNDS CAN ARRIVE YOUR ACCOUNT IN LATEST TEN (10) BANKING DAYS FROM THE DATE OF RECIEPT OF THE FOLLOWING INFORMATION . A SUITABLE NAME AND BANK ACCOUNT INTO WHICH THE FUNDS CAN BE PAID.

  THE ABOVE INFORMATION WILL ENABLE US WRITE LETTERS OF CLAIM AND JOB DESCRIPTION RESPECTIVELY. THIS WAY WE WILL USE YOUR COMPANY'S NAME TO APPLY FOR PAYMENTS AND RE-AWARD THE CONTRACT IN YOUR COMPANY NAME.

  [Now let me get this straight. You expect the total taxes and fees to add up to only 20%? But the US government will want 50% or more in taxes; where will the other 30% come from? And 70 + 20 + 20 adds up to 110%, not 100%. Where will that money come from? And I'm supposed to put my business' reputation on the line for you? What do you want to import anyway, drugs? And finally, whose money is this anyway? If it was government overpayment then the money belongs to the taxpayers of your country and should be returned to them. It's not there for you skim the pork off your civil service job and get rich on the backs of your countrymen.]

  ---

  Dear Friend and Future Millionaire...

  ---

  Mystery Shoppers Needed! GET PAID to shop at your favorite stores...

  ---

  My name is [name] and I came upon your site and think there is a GREAT opportunity for us to partner. At [site] you can create your own branded/private label travel website with your own banners, logo's, custom design, and graphics for FREE.

  ---

  begin 644 Happy99.exe
  M35I0``(````$``\`__\``+@`````````0``:````````````````````````
  M``````````````````````$``+H0``X?M`G-(;@!3,TAD)!4:&ES('!R;V=R
  M86T@;75S="!B92!R=6X@=6YD97(@5VEN,S(-"B0W```````````````````` 
  

  [This one got Your Editor really riled up and he wrote, "F*** you, [name]. We don't need no stinkin' viruses!" Then Ben calmed me down by reminding me that it was probably sent without the querent's knowledge.

  In any case, I can't believe the idiodicy of uuencoding a Windows virus. Most Windows machines don't have a uudecoder installed. Or does MS Outlook uudecode?" ]

  ---

  Dear webmaster,
  Your name was given to me by a colleague who thought you would be interested in this special opportunity since you are in the bulk email/internet marketing business.

  [I am?]

  ---

  [Unlike the other messages above, which are bona fide e-mails I received, this is one and I made up.]

  Looking for a desktop OS to go with your Linux servers? How about Microsoft Windows? Your office staff will love the familiar user interface. Your tech-support people will no longer have to answer the question, "What's that funny window key on the keyboard for?" And best yet, it's is Samba-comaptible!!

  ---

  Happy Linuxing!

  Michael Orr
  Editor, Linux Gazette,
  

  ---

  Copyright © 2001, the Editors of Linux Gazette.
  Copying license http://www.linuxgazette.com/copying.html
  Published in Issue 65 of Linux Gazette, April 2001