the boredom of ramfree17

Last weekend I reinstalled my Linux system (and not because I want to rename it as Pyg!) which is a very bad decision as it is the same time that my pitiful internet connection fails to transmit any data beyond a few bytes. The new install to Ubuntu 15.04 (Vivid Verve) gave me the confidence to start using btrfs which is being touted as the next generation file system for linux systems. Bad move and very short-sighted on my part. 🙁

The btrfs is supposed to provide enhancements to ext4 file system since it (btrfs) is built from the ground up but with the useful features already present in ext4. The ext4 filesystem is basically being viewed as an aging hack on a hack since it builds on ext3 which is a tack-on improvement for ext2. Btrfs might be delivering on some of its promised features but those who use it should still be wary of it as it is still not considered as ready for “production” or real-world use[1]. I wish I knew this before I chose it as the filesystem for my home directory. 🙁

For those that will have the misfortune of encountering it, the gist of my predicament follows (which is based on how I remember it):

• I upgraded my system using dist-upgrade.
• I encountered a severe freeze which left me with no choice but to reach for the power button.
• System bootup hangs. The last entry goes something like this:

A start job is running for /home… (10s /no limit)

• Eventually the bootup hangs and it drops the control to an emergency shell.

It was unnerving for me since a reinstall means I would need to re-download the upgrade packages on my #@$@#$ Smart LTE connection. The light-bulb moment for me was that /home is the only partition in the sequence that is using btrfs. The working theory is something borked during the forced shutdown which should have been handled by the journalling features. Thanks to an alternate albeit slow alternative connection, I was able to google enough to do the following:

• Boot the machine and press F12 after the BIOS/UEFI prompt.
• Selected the menu option for Ubuntu Advance options.
• Selected the option for “(recovery)”
• Selected the “drop to root shell” option.
• Remounted the root system in r/w mode. I dont think this is needed but this is what I have done and it won’t hurt anyhow.

# mount -o remount,rw /

• Did a btrfs file system check on the home partition

# btrfsck /dev/sda4

• When nothing popped out, I went for the repair option.

# btrfsck --repair /dev/sda4

The repair option is always accompanied by a cautionary warning as it can delete information so having a proper backup is recommended. In my situation that is like adding salt to the injury. When I ran the repair option it reported that the cache and super generation areas do not match and that it cleared the space cache. After doing so I ran another btrfsck on the partition and rebooted the system which thankfully landed me on a working system with no (identified) data loss.

The moral of the story is to select btrfs only for the filesystems that you can afford to lose data; or have a good backup scheme. Using it for /tmp is going to be an overkill but to each to his own. Now Im thinking if I want to convert my home partition back to ext4. 😀

ciao!

[1] https://www.wikivs.com/wiki/Btrfs_vs_ext4

For the impatient, jump to the end of this post for the summary of the commands to be invoked. 😉

I have signed up again for a Coursera course for Interactive Python programming to improve my Python programming skills. I am going to sidestep the question if I am going to finish this course for the moment. 😉

The course professors have developed and are using CodeSkulptor to make it easier to create, submit, and grade the course. CodeSkulptor is basically a browser-based Python interpreter that implements a subset of Python 2.x. It absolves the students enrolled in the course from the requirement to install Python in their local machines plus it also allows them to continue working on their code while on different machines (provided they know their work’s randomly generated URL).

The abstraction provided by CodeSkulptor however renders some incompatibilities in running the code locally. This could be a disadvantage for people who wants to work offline, either by choice or by location. Coursera member Jimmy Delgado has provided good quick tutorial on how to run CodeSkulptor code locally. The post is in the Coursera forums but I am replicating it here and then augment it with the steps I have added to make it run on my Ubuntu 14.04 installation.

More

Yesterday pixie, my RPi serving as the torrent/dlna box, stopped responding. Rebooting it doesnt help as it eventually reverts to only having the red LED on which I have started to interpret as the system is not yet booted.

I pulled pixie out of its nook and hooked it to my monitor before rebooting it. It showed that the boot process is encountering errors when reading from the SD card. The process stopped while asking for the root password to start the file check maintenance, or Ctrl-D to continue the boot process. I plugged a keyboard and here lies the conundrum: this is a debian system and I have been administering it as the pi user. I have been relying on the sudo mechanism and never replaced the root password so I cant provide it. That realization blows.

No other recourse now but to pull out the SD card and have the partition checked on my desktop. A “sudo fsck /dev/sdb2 -y” command (because sdb2 is the partition assigned to it by udev) and ten minutes of automated fixing later pixie is back online serving its DLNA goodness.

Now I made sure I have changed the root password so this can be fixed without booting the desktop. Come to think of it, I am doing it also on my Ubuntu desktop. 😉

ciao!

Hectic schedule but I finally finished having my own TorrentBox/DLNA/UPnP server out of the Raspberry Pis. One concern that got answered is if the system is usable as a media streaming server as there has been reports of slow throughput since the Pi’s ethernet (network) and USB system share the same memory space. Early on the file I/O throughput on the SD card I am using as the BIOS and operating system is a little bit slow, and as a techie friend (named Gideon[1]) pointed out its the class of the card itself that is the bottleneck.

The instructions are all over the net so Ill just reference the articles I used and add the other stuff I did to make it work on Pixie (the name of this Pi).
More

It was the first time to generate and send the report. It deals with downloading a lot of access logs, combining them based on certain scenarios, filtering out the cruft, slicing and dicing per service accessed and then tallying the responses to see if they meet the SLA.

The first manual extraction took around 4 hours to create the report for the most critical service. After that, I sat down and created the bash+awk script to do the slicing, dicing and tallying part. It took me 3 hours to create and test the script but after that I can generate the reports for 6 services in 30 minutes, where the bulk of the time was spent in downloading the latest logs (~10minutes) and pasting the data in Excel to create the “bayoootiful” graphs.

I figured if I can automate the rest then the reports can be generated under 15minutes, and I can hook it up on a continuous service so it will send it automatically every night. After that I can work on getting rid of the need to go through Excel if I can find some API to generate it and publish them as PDFs. JFree perhaps but my Java skills are already rusty. I also need something native if possible as I cannot install stuff on the office machines. The zLinux servers are pretty much off limits though I think there is a python interpreter installed.

One thing I can say is that automation and f/loss rocks. The script is too specific and covered by IPR so I cannot post it here but it is something that most scripters should be able to do.

ciao!