Bashwk to basics

No Comments

It was the first time to generate and send the report. It deals with downloading a lot of access logs, combining them based on certain scenarios, filtering out the cruft, slicing and dicing per service accessed and then tallying the responses to see if they meet the SLA.

The first manual extraction took around 4 hours to create the report for the most critical service. After that, I sat down and created the bash+awk script to do the slicing, dicing and tallying part. It took me 3 hours to create and test the script but after that I can generate the reports for 6 services in 30 minutes, where the bulk of the time was spent in downloading the latest logs (~10minutes) and pasting the data in Excel to create the “bayoootiful” graphs.

I figured if I can automate the rest then the reports can be generated under 15minutes, and I can hook it up on a continuous service so it will send it automatically every night. After that I can work on getting rid of the need to go through Excel if I can find some API to generate it and publish them as PDFs. JFree perhaps but my Java skills are already rusty. I also need something native if possible as I cannot install stuff on the office machines. The zLinux servers are pretty much off limits though I think there is a python interpreter installed.

One thing I can say is that automation and f/loss rocks. The script is too specific and covered by IPR so I cannot post it here but it is something that most scripters should be able to do.


Using SMART BRO (USB Internet) Plug-it in Linux


This guide is created to show how to use the SmartBro USB Internet dongle in Ubuntu Linux. The dongle comes with the Windows installer that enables the SmartBro application to be installed in most computers. From within that application the user is able to send and reeive SMS, and connect/disconnect their Internet subscription. It allows more operations like access to the Smart portal but for the duration of this guide only the SMS send/receive and Internet configuration is covered. The instructions should be portable to the plug-it kits of the other telcos.

From my experience, Ubuntu has easily supported the USB dongles and tethering devices as early as Karmic Koala. I no longer have my Sony Ericsson K618i that I just connect via a USB cable to my MSI Wind U100 Netbook so this is what I will use for the guide. My netbook is sporting Natty Narwhal during the time the screenshots are taken.


Lets get cracking. Caveat though that I had to recreate some activities in two different locations so don’t mind the clock too much.

Configuring data connectivity

  • Boot into Ubuntu and connect the dongle. Wait for a few second while the system recognizes the device.
  • From the system tray, click on the network-manager applet and enable the “Enable Mobile Broadband” option. Select the “Edit Connections…” entry, and add a new entry under the “Mobile Broadband” tab.
  • As can be seen below, Ubuntu has already detected the chipset of the USB dongle. Click on the Forward button.
  • Select the provider (which in this case is Smart). Click on the Forward button.
  • On the billing plan dialog, accept the default selection and click on the Forward button.
  • Confirm the summary of the selected settings by clicking the Apply button.
  • Visit back the settings of the created connection and ensure that there is no username and password in the Mobile Broadband settings page. Save any changes made and dismiss the wizard to go back to the desktop.
  • Click on the network-manager applet and click on the mobile internet entry created above.
  • If everything goes well and the signal is strong enough the user should be greeted with an established connection message.
  • Click on the network-manager to validate the type of connection.

Configuring SMS capability

The setup above can fail if the SIM card used is of the prepaid type and there is no credit loaded on the account. SMART imposes a Php10/30mins rate but there are promos that can be activated by sending some keywords via the SIM account. This implies that the SMS send and received facility should be accessed. Fortunately the wammu project provides this capability.

  • Install the wammu application by invoking the command below:
  • sudo apt-get update && sudo apt-get install wammu -y

  • Start the wammu application. If the wammu application doesn’t find its configuration files then it will trigger a wizard for first time-configuration.
  • wammu-first

  • Click on the Next button to start the Wizard and select the following options in the succeeding windows:
  • – Configuration Style: Guided Configuration
    – Configuration type: USB Cable (as the unit is plugged in a USB port)
    – Phone Type: None of the above (unless future versions provide an option for the chipset)

    – Connection Type: AT based
    – Driver to use: Generic AT over serial line or it’s emulation
    – Phone device: /dev/ttyUSB0

    The above part is trial and error. If the test doesn’t succeed select the next option. If it succeeds then information about the model of the dongle will be shown. In my experience it is almost always /dev/ttyUSB0 but then I do have another device that emulates a serial connection via USB.

    – Complete the remainder of the next dialog windows using the default option.

  • Once the wammu phone configuration is done, start accessing the USB dongle by invoking the Phone->Connect menu item.
  • Once the dongle is active, test the connectivity by receiving existing messages.
  • After the operation succesfully completes, select the Messages item in the left treebox to view all retrieved messages.
  • Next is to try the SMS sending capability under the Create menu. The editor is dated compared to the built-in provider interface but it would suffice. In the screenshot below, the bucket promo of Always On is being triggered.
  • Check the Sent folder if the message was successfully sent.
  • After a few seconds/minutes, retrieve the messages again to check the result of the promo registration.

And that is basically it. The instruction in this guide, with minor tweaks, should be applicable to any of the USB dongle. Enjoy your USB Internet dongle in Linux. 🙂


Faster file copying over a local network

No Comments

What I started to do was to copy a 460MB file from the computer in the project network into my laptop before I leave in an hour. The problem is that when I started mapping the drive containing the file and copying it using Windows explorer the ETC displayed by the file copy dialog box fluctuates between 2-5 hours. This instability and slowness is a pet peeve I have against the Windows copy process over CIFS/SMB shares. I know copying from the command line is going to be faster but without a progress meter I don’t know when it is going to be finished.

The next option I did was to share it the webserver hosted in the remote box and then tried to download it. The download speed range is between 30-75kbps so the ETC reported by the browser is between 2-3 hours. It may be faster but not fast enough for my needs. Then I realized that I have rsync inside the cygwin install in the laptop. As I have already the network share mapped to Z: drive so the simple command

$ rsync -avp --partial --progress /cygdrive/z/Temp/ ./

started the transfer which was a whopping speed range of 230-420Kbps. ETC is now 15-20 minutes. Problem solved so I am writing this blog while waiting for the transfer to complete. For those interested with the rsync options:

  • -avp : copy the file in archive mode, preserve permissions and increase verbosity.
  • –partial : if the error fails, don’t delete the transferred chunks so it can be used for resuming the transfer.
  • –progress : show the copy progress.

Why didn’t I just use a USB stick? It is because the company has locked down the USB storage drivers and while somebody who amazingly has a strong resemblance to me can do something about it, it would be too troublesome to pursue. FTP might have been a good choice of protocol as this is a secured environment but from experience the antiquated file transfer protocol only has a slight advantage over HTTP in terms transfer speeds.


Semi-hosed Wind UMPC (Fixed)

No Comments

Fixed by overriding the GRUB lines, editing the GRUB menu, and correcting the entries in the /etc/fstab.

Here is the original line that causes the bootup to panice:

kernel (hd1,0)/boot/vmlinuz BOOT_IMAGE=linux root=/dev/sdc10 vga=788
initrd (hd1,0)/boot/initrd.img

PCLinuxOS is configured to use the graphical GRUB menu so pressing the ESC key will allow the users to go into text mode that allows the editing of the line command. Pressing ‘e’ on the selected entry enabled to correct the order of hard drive being searched [ (hd0,0)/boot] as well as the correct root [root=dev/sda10]. After both the kernel and initrd lines have been updated, pressing ‘b’ will continue the boot process.

The GRUB entry is only half of the problem since the boot operationg still panics and dumps me in a command line. The culprit is that the fstab file still contains incorrect entries since all of the partitions are being searched at /dev/sdc. The USB flashdrive I used for the installation has 3 partitions so there are also 3 incorrect entries for it in /etc/fstab.

As I said, the boot process dumps me into a command line. This is where knowledge of vi comes very handy. It is small enough and ubiquitous enough to be in most *nix system. A quick ‘vi /etc/fstab’ to correct all linux partitions (and temporarily disable mounting of windows partitions) and I am now good to go. Invoking ‘mount -a’ confirms that I am able to successfully mount all partitions.

This problem is solved but unfortunately the PCLinuxOS Minime distribution is not the answer to my wireless in the Wind woes. 🙁


Semi-hosed Wind UMPC

No Comments

Doc Mana (or rather his daughters) own a Wind and has pointed me to the released Linux realtek drivers. As Doc Mana has stated, the released drivers are in RPM format but this gives me a good opportunity to use the alien utility for converting packages. Doc also said his Neo empriva is working fine with a similar wireless chipset.

With the extended weekend, I promised to try it out with a refreshed Xubuntu installation, or better yet try installing the PCLinuxOS MiniMe distro to get native support for RPM. The nutshell is that I now have a semi-hosed Wind UMPC. 🙁

Here is what happened:

  • I replaced the gnome network manager with the kwlan application and I hosed down my network connections. I then decided to refresh the Xubuntu installation from the my 4GB flash disk. And then…
  • I tried converting the package using the alien utility with the “–scripts” parameter because it told me so. It said it is for including the scripts inside the package. When it was time to install them with dpkg, an error was thrown out that there is a parsing error in the post install script. Both drivers show this error. Great!
  • As this is a refreshed installation, I decided to try again the tip from the wiki as this is also stated in the Gentoo Wind wiki. The build essential tools refused to install properly because of the problem with the botched realtek drivers installation.
  • I cannot uninstall/purge the realtek drivers from synaptic becase there are errors in the hooked scripts. 🙁
  • I wiped by flash disk with MiniMe and proceeded to install over my Xubuntu system. Everything seems to be fine until I rebooted. The system got reconfigured with my flash disk as its main storage (sda) and my Wind’s drive as its secondary (sdc) so the GRUB entries refuses to work.
  • I tried booting with the flash disk inserted and tried to select the HDD Minime installation but the boot process refuses to proceed and ends with a kernel panic.

Right now the Wind will only continue booting if I have the flash disk inserted, and its too late into the night to start tinkering with livecd options. The bright side is that I have effectively created a very crude security key. /lol

I hope I can fix this properly tomorrow as the wife is giving me weird looks when she can’t easily get into the Windows partition to play Mystery Case Files. 😀


Older Entries Newer Entries