Bashwk to basics

May 07

ErinFLOSS, Work , , No Comments

It was the first time to generate and send the report. It deals with downloading a lot of access logs, combining them based on certain scenarios, filtering out the cruft, slicing and dicing per service accessed and then tallying the responses to see if they meet the SLA.

The first manual extraction took around 4 hours to create the report for the most critical service. After that, I sat down and created the bash+awk script to do the slicing, dicing and tallying part. It took me 3 hours to create and test the script but after that I can generate the reports for 6 services in 30 minutes, where the bulk of the time was spent in downloading the latest logs (~10minutes) and pasting the data in Excel to create the “bayoootiful” graphs.

I figured if I can automate the rest then the reports can be generated under 15minutes, and I can hook it up on a continuous service so it will send it automatically every night. After that I can work on getting rid of the need to go through Excel if I can find some API to generate it and publish them as PDFs. JFree perhaps but my Java skills are already rusty. I also need something native if possible as I cannot install stuff on the office machines. The zLinux servers are pretty much off limits though I think there is a python interpreter installed.

One thing I can say is that automation and f/loss rocks. The script is too specific and covered by IPR so I cannot post it here but it is something that most scripters should be able to do.

ciao!