Command line tool for Internet Usage Monitoring in Linux

Yeah, a command line tool for monitoring the internet usage so that I can avoid over usage of the internet. This was something that I have been trying to develop for quite sometime and the breakthrough happened just yesterday.  Initially I was trying to develop it using  shell scripts. The whole script was centered at the ifconfig command. When I tried installing it to the crontab, it won't work for some reason which I don't know yet. Then I turned to Python. Here too the script was centered on the ifconfig. But to my disappointment that too didn't work when installed to crontab. it was at this point that I thought about a different strategy. In order to get the details of Internet usage in current session I decided to rely on the file, the ifconfig command is relying rather than the relying on ifconfig command itself.
                       So the next challenge was finding out the file ifconfig is relying. Then I decided to analyse the system calls used by ifconfig . Here I took the help of strace command and arrived at the conclusion /proc/net/dev is the file that is used by the ifconfig for the details of internet usage in current session. And with this file things become lot easier.
              As of now this application will monitor the usage for one month. Then it is reset to zero automatically. You can change the day of the month in which the monitoring begins. The application  also has a feature wherein  you can set free usage time, so that the internet usage is not monitored between the specified time.It can only monitor the internet communication taking place through eth0 (wired) eth1 (wireless) interfaces. 


            You may track the development here.  You can install it by running Installer.sh script (Don't change the file names and all files should be in the same directory as of Installer.sh ). After installing you can use the tool by using the command Eusage along with the switches.
Note : Scripts won't work without running Installer.sh. Don't try to run Eusage without the switches
Eusage -u :- used to give the details of usage so far, from the beginning of the month 
Eusage -r :- To clear the memory (i.e. everything        will be set to zero)
Eusage -h :- for help
Eusage -c :- for changing free usage time and to set the day of month to begin the monitoring

Twitter for System Administration

How about shutting down/restarting your system via SMS? 
Is that something that is possible? Yes , With twitter SMS service that is of course possible. This can come handy for system administrators. Here I will show you how to control your system  on the move via SMS, using the services provided by the twitter.
          Initially we will create two twitter accounts , call them Master and Slave. Master account will be a normal account, on the other hand Slave will be a private account, which has connection only with Master(i.e Slave will be only following the Master and the only follower that Slave has, will be Master). This helps preventing users other than Master sending messages to the Slave.
           The next step will be a program that monitors the message inbox of the Slave for the messages from the Master. This program have to be run on the system that has to be controlled. The program should be able to log in to the account of Slave and to check out its inbox. That's where twitter API comes to rescue. For various reasons twitter prevents earlier Basic authentication. So we have to rely on OAuth . OAuth is a trickier authentication method , whose explanation is beyond the scope of this article. Using twitter api resources we will be checking out the message inbox of Slave. Remember Master is the only user in twitter who can send messages to the Slave . Also message to a twitter user can be sent via SMS .Messages are monitored by the program and depending on the message, necessary actions are taken. Below is simple program that will Shut down/restart the system depending on the message. 


Though the sample program can only be used for Shutting down/restarting the system ,System Administrators can add the functionality like auto replying to the queries issued by the Master, thus achieving a greater control over the system via SMS.
Note : Python tweepy module is a module that helps in communication with twitter. It can be installed in linux system by the command easy_install tweepy. Also the program is installed in crontab ,so that it runs every minute.

Calicut University result summary using Shell & Python scripts

 I wanted to create the summary of the university exam results of my class with details like ranklist, the number of students between different SGPA limits and the subject wise performance of students. I developed Shell script for the first two procedures and a Python script for the first and last procedures. Both the scripts involves the process of crawling through individual results of students. Here is the link

SHELL SCRIPT
 wget  & pdftotext  are the most important tools I have used in this shell script. wget is a tool that can be used to download web pages. Calicut university website prevents the user-agent (eg. wget ) programs from accessing it. So in order to access the result pages , spoofing is required. In that case we will make the website believe that its a browser that's actually accessing it instead of the wget program.This type of spoofing can easily done by the command : wget -U browsername URL. So that is how we downoad individual result pages...

Tip :    wget when used with -r switch will download the web pages recursively.

Calicut university  on its part publishes the individual results in PDF format. Since creating summary requires text processing, it is important that the PDF files are converted to text format. That is where pdftotext comes handy. As name suggests , it is used for converting PDF format to text format files. After downloading and converting the individual results  to text , the summary is just a matter of some amount of text processing.

PYTHON SCRIPT

                       
                  In order to  crawl through individual result , I took the help of Python urllib module.Challenge in this script too was converting  PDF file to text file. Here Python pdfminer module helped me. Along with the PDF miner, a program pdf2txt.py gets installed. This program can be used to convert the pdf file in to text file. As stated earlier the remaining tasks is just all about text processing.