110GB+ worth of photos/videos on a microSD memory card #2

In my previous post, I talked about wanting to write a script to see how far I can squeeze 110GB worth of media (photos and videos).

I wrote the script the quick-and-dirty way, in PHP. When people think of PHP, they think of web pages, but PHP can equally be used for writing your everyday scripts, just like Perl or Python.

The script resizes all photos to a maximum width/height of 1024. I found this provides maximum size reduction for the photos without sacrificing the viewing quality in full photo mode on my tablet (screen resolution: 1280x800). Obviously it will affect the quality when zooming in, but I rarely do that when flipping through the photos. Since I haven't got a tablet with retina display yet, I am not sure how that will fare, but for this exercise, I am sacrificing quality for size.

For videos, I use handbrake-cli to transcode all of them to MP4 with a maximum width of 640. Again, this reduces the size drastically, improves compatibility (since all Android tablets/smartphones I have come across so far can play MP4 with no problems) and looks OK quality-wise on my tablet.

The script takes a source directory and replicates the folder structure in a target directory. It transcodes all files with certain extensions (configurable in config.php) and places them in the corresponding target subdirectories. If the file has already been transcoded, it will not be transcoded again, so it will be relatively quick to update the target directory after adding more files to the source directory.

The script took 1.5 days to transcode my 110GB collection (bear in mind this is a older, slower AMD Turion II laptop). The result was only 29GB and fits easily into a 32GB microSD card.

I popped the microSD card into my tablet and viewed the files using QuickPic. The result was awesome.Check it out in the video below:

Note that this video was produced with the microSD card inserted into an al-cheapo single-core tablet bought for the kids during a sale. I was pleasantly surprised by how quick and smooth it was. Needless to say, on my own 10" quad-core tablet, the result was even better.

QuickPic is a better app than Gallery for this purpose because:

1. It allows you to set "Include Folders" to include only the media files from the microSD card

2. It lets you set the 'Explorer View" to browse through the folders/subfolders in a hierarchical manner.

3. It has "Stack", "Grid" and "List" views for different visual summaries of the folder content.

4. It is quick and has tons of other options.

The result of the transcoding operation exceeded all my expectations. I have a snapshot of 110GB worth of media on a tiny memory card, ready to be popped into any tablet or PC to view. At only 29GB, there's lots of room for growth, especially on a 64GB microSD card (which no doubt all tablets will support in the future, even cheapo ones).

If you think you might find this useful, you can download the script below. It includes the PHP and Handbrake-CLI binaries so it should be ready to run under Windows. It shouldn't be too difficult to adapt the script to work under other platforms as well.

After download, unzip into another directory and edit config.php. You only need to change $srcdir and $tgtdir. Then bring up a command prompt and run transcode.

Once you verify that everything works correctly, you may want to schedule it to run periodically via the Task Scheduler.

Download: Batch Media Transcoder V1.0 (ZIP 25.2MB)

Download: Batch Media Transcoder V1.0 (7-ZIP 17.1MB)

Read More

110GB+ worth of photos/videos on a microSD memory card #1

We have a family NAS where we keep all kinds of stuff. Well, actually it's an old laptop that functions as a server, with 3 x 3TB external USB HDDs attached. I run all the standard mirroring and backup on the laptop, and one of the HDD is shared over the network for everyone to dump their stuff into.

Like most family these days, we take photos and videos on a number of different devices. The smart phones, digital camera, DSLR etc. Even the kids have their own cheap digital compacts. And the devices have evolved over the years. The photos have mostly stayed JPEG, with the resolution increasing steadily. The videos, however, are a different story. Starting from the V8 (which we converted to XVID AVIs) to a Sony (which produced MPEG-1 files) to a Casio (which gave us H264/MOV files) to the current-gen Sony (MTS format), there's an incredible number of video formats we have to deal with. On top of that, we do scans of old photos and slides, the kids' artwork etc. These are all dumped into the family NAS.

To avoid being locked into a particular platform, we resisted moving to any software that relies on an underlying database for organizing the photos. Instead we created folders (2010, 2011, 2012 etc) and subfolders (eg. 20100825 Adam's Birthday Party) within the underlying filesystem to organize the photos. With the subfolders, the photos and videos are renamed to "YYYYMMDD-hhmmss" format. jhead is a great help batch-renaming of photos. For video files, the strategy varies. And not all filles can be renamed this way (eg. the scanned photos). Though crude, this has mostly worked. It is also quite flexible when it comes to backup and batch processing.

For viewing the photos, we use XnView.

IrfanView works as well, but it requires more tinkering to get it working with the different video file formats that we have. XnView works pretty much out of the box with the video files.

These two apps have a number of advantages going for them:

1. They are both freeware with lots of plugin support.

2. They work at the filesystem level, and we can easily expand/collapse the folders/subfolders to view our media archive. Some apps require you to import the media that you want to view.

3. They display both photos and videos, both at the thumbnail and individual media level. Most media viewing apps support only photos.

4. They are portable, meaning they can be easily copied/moved from machine to machine without re-installation and executed with their settings intact.

The downside is that both apps are not particularly sexy, but they do get the work done quickly and effectively.

One problem I face is that the photo/video archive on the NAS is standing at 110GB and growing steadily. I want to be able to view this archive on my Android tablet, which is a very natural medium for perusing all the photos and videos that we have accumulated. But fitting 110GB of files into the tablet is next to impossible currently (the largest microSD card is 64GB), and even if I could, the tablet won't be able to deal with all the video formats in the archive (the 1080p MTS files are particularly problematic).

So I decided to write a script to transcode all the files in the archive to a smaller size to see how much I can squeeze it down without ruining the viewing experience on the tablet. This will be the topic of my next post.

Read More

Coding the Liturgical Calendar

For a long time, I've wanted to integrate the Roman Breviary into a couple of projects (nothing public) but have never been able to find a suitable calendar, breviary or psalter implementation.  So I wrote one in python.  It requires the `dateutil` package, but other than that its standard python.  My principle use of this is going to be to create a JSON interface to get the current office, season, feast-day and reading from the 1962 Liturgical calendar.  I am working on a "modern" liturgical calendar, its just not my priority right now. 

Read More

Fast Diet - Hunger Pangs

When I am doing the Fast Diet, I frequently experience long periods of calm after a brief period of hunger pangs. It is as if the body has gone into hibernation, and I am left well alone to carry on with my daily routine. This is most apparent the morning after the diet day, when I could usually skip breakfast and have a late lunch.

I am reminded of what my diving instructor told me during training many years ago, that feelings of suffocation after a brief period under water is not due to lack of oxygen, but due to the buildup of carbon dioxide in the blood stream. This means you are not going to die (yet), and with knowledge you should  try to suppress the panic, remain calm and find a way out of the current situation to surface.

Hunger is mostly the same thing. It makes us uncomfortable and compels us to eat to rid ourselves of the discomfort. Worse, we have come to associate a lot of other feelings (both positive and negative) with eating. So we eat to make ourselves feel better, not necessarily when we are hungry. And this slowly builds up over the years in the world we live in today, where food is available in abundence 24x7.

I think occasional calories restriction (which is essentially what the Fast Diet is about) serves to remind ourselves that it is OK to feel hungry, that you don't have to eat at the first sign of hunger. It also serves to calibrate the "gauges" in our bodies and I think that's where the health benefits are derived.

Read More

Looking for the perfect car MP3 player/FM transmitter

I have bought quite a number of car MP3 player/FM transmitters from DealExtreme and other similar websites.

A car MP3 player/FM transmitter is a device that takes a SDHC memory card or a USB memory stick and plugs into the car cigarette lighter socket. It plays the MP3 files stored on the memory card/stick and transmits that as FM signal to be picked up and played by the car stereo.

It is a nifty and convenient device because not all car stereo can play MP3 files, or be able to accept a memory card or stick. Also such a device is very portable. You can simply plug it into any vehicle eg. a company car and it will play all your favorite music.

Other alternatives exist eg. bluetooth, but they are simply not as straightforward and universal as this solution. After all, you'd be hard-pressed to find a car without a cigarette lighter plug.

The perfect car MP3 player/FM transmitter should have the following characteristics/features:

1. Emits strong FM signal
2. Remembers last FM frequency used
3. Remembers last MP3 file played
4. Remembers position of last MP3 file played
5. Supports Unicode font on LCD display
6. Body of memory card should not stick out when plugged in

1 and 3 are pretty much essential for the device to be useful. Unfortunately I have had to throw at least .a few devices away right upon arrival because they were lacking one or both of these features.

2 and 4 are good-to-have features (some may even consider them essential). 2 can be overcome by setting the car stereo to the default frequency, but sometimes you need to change the default frequency on the device because of interference, and having to do that during every cold start gets old after a while.

4 should be quite easy to do (just a little extra memory to store the last play position), but surprisingly I have not seen any device that has this feature! Either there is some unknown technical barrier to implementing this feature, or Chinese manufacturers are .too stingy to spend a few more cents to do so.

5 is a nice-to-have, since none of my Mandarin songs have their ID3 displayed correctly. On the other hand, this is understandable because it will probably add a lot to the production cost to embed a Unicode font into such a tiny device.

6 is simply a design issue. Half of the devices have it right, others have the memory card sticking out so badly I wonder what the designers were thinking.

I have yet to find my perfect car MP3 player/FM transmitter. I have bought at least a dozen such devices now, and none of them have all of 1,2,3 implemented, not to mention the others. The local Chinese OEM devices sold in Oz stores are not .much better.

Anyway, if you know of the perfect car MP3 player/FM transmitter, please leave me a comment. I am willing to pay a premium for such a device, but I am tired of buying one device after another and being disappointed over and over again...

Read More

Children - The ultimate test in anger management

Having just spent two full holiday weeks with my kids, it occurred to me this morning that raising young children is the ultimate test in anger management.

You know the lines. Address the situation, not the other person. Bring change to yourself first, not to the other person. Easy on paper, very difficult in practice.

With adults, it is often easy to blame it on the other person. After all, he/she is a full-grown adult, why can't he/she be more mature, rational, have a good grounding in the facts etc.?

With children, you can't pin the blame on them anymore! They are what they are at this age. You can't expect them to be mature or rational! But we still lose our cool, and that's a perfect reflection of our own emotional management skills.

Therein lies the ultimate test in anger management. We have to change ourselves to address the situation, and that ain't always easy,  because being human means we resist change with all the might we could muster.

Failure to manage our anger eventually leads to child abuse. The male gender is more frequently associated with such behaviour, but it seems the female gender is sometimes not spare such emotional rampage too.

Our children grow up with us, but we grow up with our children too. Good luck to all the parents traveling on this amazing journey together!

Read More

Fast Diet - Incredible Body Hack

In terms of body hacks, the Fast Diet probably ranks all the way at the top for me.

I first read about it in the papers and found the technique rather intriguing. Could something that simple actually work? Besides the book, it didn't really have anything fancy to sell. No secret formula, patent-pending ingredient, celebrity endorsement etc.

A few weeks later, at a whim, I started to try the diet on myself. At this point, I have not read the book yet. I just started restricting my diet two days a week to 600 calories. And I wasn't too particular with counting the calories either. I simply looked it up on the smartphone and kept a rough count through the day. Sometimes I prepared the food specifically for myself. Other times, I simply ate a smaller portion of whatever the wife had cooked. One or twice, I actually broke fast midday as unexpected commitments cropped up.

I also wasn't particularly keeping a keen eye on my weight during this time. Ironically, I didn't really think the diet would work. It just seems too free-and-easy. Not enough suffering! Then about 8 weeks later, I weighed myself in the bathroom and found that I had lost 6kg. To say I was surprised was an understatement! I started weighing myself more regularly from then on, now out of curiosity then anything. I found myself steadily losing weight at a rate of about 0.5kg per week. Another 8 weeks on, I have lost another 3kg or so. I haven't seen this weight for a long long time, not since my university days!

And I was still eating all my usual junk on the off days: cakes, chips, chocolate, fries, fizzy drinks. Nothing "health conscious" has crept into my mindset so far that I was deliberately avoiding them. I was just eating normally as before.

I did pick up the book to see if I coud learn anything new, but I didn't. I am quite familiar with scientific studies of how severe calorie restriction can prolong life, but most of us can't live that way. Even ADF (Alternate Day Fasting) seems too harsh to me. I think the breakthrough for the book is to introduce the idea that even a moderate amount of fasting will do us plenty of good. Other than that, I am not sure one has to read the book in order to get started on the diet. I don't even find the recipes in the book particularly useful. As I mentioned, I did quite alright with eating whatever my wife cooks, but in smaller quantities.

The book did mention that once you reach the desired weight, you can go into "maintenance mode" and reduce the fasting to once a week. For me, I plan to carry on fasting twice a week just to see how far this diet can hack my body weight. I am pretty sure my weight will stabilize at a certain point and will not reduce indefinitely! I will blog about my progress in times to come.

Read More

ComboFix - Malware Removal Tool

Again, I am writing this down more for my own future reference, with the hope that it will benefit others Googling for this info as well.

ComboFix is a relatively unknown but surprisingly good anti-malware tool. I had in the past received a number of computers from relatives/friends that have been infected with malware. The general characteristics are that the machines boots OK into Windows, but runs erratically eg. displays fake error messages, prevents you from running certain executables, logs you off after awhile etc.

In my experience, it was impossible to get the existing antivirus software to perform a proper scan, or to install a new antivirus software because the system ran so erratically. Offline antivirus software that runs off a CD didn't help much either. I have a collection of them and none of them had helped on the computers I was trying to fix.

Enter ComboFix, which was able to get the computers back into good enough working condition so that I can reinstall the antivirus software and perform a proper scan. I am not sure what's the exact magic behind the program. It seems to "implement a collection of pre-made fixes for large amount of known malware and hunts down all files associated with it". Sounds good to me!

Windows Repair (All In One) is also helpful after running ComboFix. Some malware messes up all kinds of stuff, and functions like "Set Windows Services To Default Startup" is invaluable for straightening the system.

Of course, some people will tell you it's best to wipe the machine when it has been compromised. From a technical point of view, I would totally agree and will do it with my own machines. But in the real world with real people, that's not always the ideal solution for them. So I am glad ComboFix and Windows Repair AIO has enabled me to help some people along the way.

Read More

Do not upgrade to CM10.1 if you use Bluetooth A2DP extensively

I flashed my Nexus S to CM10.1 recently and found that the A2DP streaming stutters so badly to the point of being unusable. Some googling reveals that from Android 4.1 to 4.2, Google swapped the Bluez stack that was used in Android since 2.x and replaced it with a stack contributed by Broadcom. This replacement has apparently created more issues than it has solved.

Anyway, I am back to CM10 (based on Android 4.1.2 with the Bluez stack), and all is well again. So if you use A2DP as much as I do (I listen to podcasts, 1~2 hours a day), you have been warned!

Read More

Samba Server on Android Media Player

I recently bought an Android-based media player: the Minix Neo X5. It came with the original firmware, which had issues playing the MTS video files that my digital camera produces, so I promptly upgrade to the latest ver 011 of the firmware. That solved the MTS playback issue, but the firmware itself was not very stable and hang quite frequently. So I promptly installed Clockmod Recovery, followed by the CWM version of Finless V1.2b. Now everything is fast and stable!

Next, I needed an SMB server on the media player so that I can manage its content from other PCs in the house. Surprisingly this turned out to be harder than I thought given the hackish nature of Android.

SambaDroid was quite easy to install and work with. Unfortunately, it does not support files > 2GB in size. This is quite a major limitation given than many media files these days are > 2GB in size.

Samba Server turned out to be a dud. Despite having root access, I couldn't set it up so that it is accessible by other Windows PCs. I even performed some registry hacks to enable legacy NTLM without any success.

I had partial success with Samba Filesharing (V1.0.6) and it supports files > 2GB, but I could only get it to work occasionally after a lot of coxing. Turns out it does not yet support Jelly Bean, so the app crashes the other 99% of the time, even with the samba-rc hack.

With no other candidates to try, I tried hacking into Samba Filesharing to see if I can make it work better with JB. Armed with adb shell,  I found that the Android version of smbd and nmbd are found under /data/data/com.funkyfresh.samba/files and can be started by running:

> sh ./samba-rc restart

But the app itself interfered with the operation of the daemons. Perhaps it is trying to control them and ended up crashing/killing the daemons. Anyway, the conclusion is that there is a fully working version of Samba included with the app. It's the app itself that is incompatible with JB. The app is a wrapper that receives events so that it can start smbd and nmbd when the network is up, and kill the daemons upon user request or when the network goes down.

So what I ended doing was:

1. Install and configure Samba Filesharing. This properly configures samba-rc and smb.conf.

2. Perform adb pull /data/data/com.funkyfresh.samba/files. Then update samba-rc by adding a single line.

3. Uninstall Samba Filesharing. This gets rid of the app itself so that it does not interfere with the daemons.

4. Manually create /data/data/com.funkyfresh.samba/files via adb shell.

5. Zip up the files and upload via adb push. The zipping is necessary because adb push can only handle individual files. Then adb shell and unzip.

6. Perform chmod 0777 samba-rc smbd nmbd.

Then I try to manually execute sh ./samba-rc restart. This started the smbd and nmbd daemons and I was able to connect to the shares from another machine.

Now this left the problem of running the script when the machine starts up, after the network interface is brought up. The simplest way I found was to use Script Manager. There are probably other ways (eg. via DHCP hooks), but I am lazy and went with the simplest method that worked.

I am sure Samba Filesharing will eventually be updated to support JB fully. There is a very active thread here on xda-developers for this purpose. But in the mean time, if Samba Filesharing does not work directly on your device yet, I hope my experience here is of some help to get it working on yours.

Read More

General (lossy) method for normalizing volume of any video file

I have an MKV video file (H264 video, AC3 audio) whose volume I need to normalize. Unfortunately, there doesn't seem to be any lossless way of normalizing the AC3 audio track. So I found out the general (lossy) way of normalizing the audio track of any video file is:

> ffmpeg -i input.mkv -vol nnn -vcodec copy output.mkv

nnn = 256 (no change)
nnn = 128 (half the volume)
nnn = 512 (double the volume)
etc.

This should work for any video file format that ffmpeg recognizes.

In addition, I can use the -map commands to remove any unwanted subtitles.

> ffmpeg -i input.mkv -map 0:0 -map 0:1 -ac 2 -vol nnn -vcodec copy output.mkv

This tells ffmpeg to only include track 0 (video) and 1 (audio) in the output file and ignore the rest.

Read More

Adding "Stereo Mixer" to Windows 7 with Conexant sound card

This procedure worked for my laptop (Thinkpad E530) with a Conexant 20671 sound card, but I suspect it will work for other sound cards in the Conexant family.

I was playing with CamStudio to do a video capture of a Flash-based cartoon so that I can put it on the WDTV media player and play it on the big screen in the living room for my kids. The video capture worked brilliantly, but to do a sound capture, I needed to do some hacking.

Apparently, there was this recording device called "Stereo Mixer" that was pretty standard in the Windows XP days. This allowed you to capture whatever was played to the speaker in all its digital glory. Then under pressure from various organizations on the dark side of the force, Microsoft and soundcard makers starting disabling this wonderful feature from Windows Vista onwards.

So after much Googling around, I found out that for most sound cards, the hardware feature is still there, just not enabled on the software side. Unfortunately, to enable the "Stereo Mixer" feature is not as simple as ticking a checkbox. The procedure is frequently complicated and is different for various soundcard chipset families.

Luckily, for the Conexant sound card on my laptop, the procedure to enable the stereo mixer feature was comparatively straightforward.

First, right-click on the "Sound" icon in the system tray and select "Recording devices":

 

Then right-click on the whitespace and select "Show Disabled Devices":

Make sure you don't see a "Stereo Mix" device that was previously hidden.

Now, you need to add some entries to the registry. Create a ".reg" file with a text editor with the following content:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\{4D36E96C-E325-11CE-BFC1-08002BE10318}\0000\Settings\EPattributes\EpSettings\StereoMixEnable]
"Enable"=hex:01
"MixAssocSeq"=hex:e0,e0
"MuteGainSettings"=hex:00,00

Double-click on the .reg file and enter it into the registry. Verify by firing up "regedit".

This enters the "StereoMixEnable" values under "HKLM\System\CurrentControlSet". You also need to enter this into all "ControlSet00x" subtrees under "HKLM\System". So for example, since I have "ControlSet001" and "ControlSet002" under "HKLM\System" on my machine, I edit the .reg file to:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Class\{4D36E96C-E325-11CE-BFC1-08002BE10318}\0000\Settings\EPattributes\EpSettings\StereoMixEnable]
"Enable"=hex:01
"MixAssocSeq"=hex:e0,e0
"MuteGainSettings"=hex:00,00

and double-click to enter it into the registry. I repeat for "ControlSet002".

Now reboot the machine. Then follow the previous steps to bring up the recording devices. If all goes well, you should now see a new, disabled "Stereo Mix" device in the list.

Right-click on the device and select "Enable":

Then set it to the default device by highlighting it, and selecting "Set Default".

Now, under CamStudio, I select "Options, Record audio from microphone". Since the default recording device is now "Stereo Mix", anything that is played through the speaker is now piped through to CamStudio for recording.

Mission accomplished!

Read More

Using Google Dashboard to manage your Android device backup

I used to use AppBrain/Fast Web Install to keep track of which apps I have installed on my phone, and to make it easier to reinstall those apps when the phone gets wiped or replaced. But AppBrain had been going down the tubes, and Fast Web Install had always been a hit-and-miss affair.

Android's own "backup to the cloud" system had previously been even more unusable. There isn't a place where you can see what has been backed up. And when you setup a new phone with your Google account, you just have to wait and pray that your favorite apps will be restored to the phone. Typically all the stars have to be aligned just right for this to happen. More often than not, after waiting for an hour or so and nothing happens, you just curse under your breath and proceed to install your favorites apps manually via the Play Store.

But I just looked again recently and was pleasantly surprised that things are much more civilized now. Firstly there is a place now where you can look at all the apps that are backed up. It's called the Google Dashboard. This is where you can find out almost all the information Google stores on your behalf across its wide range of services, including Contacts, Calendar, Docs, Gmail, Google+ etc.

If this is the first time you are visiting Google Dashboard, you will be shocked how much Google knows about you. If you are not (and you own an Android smartphone with location history turned on), then you should definitely take a look at "Google Location History" (under "Latitude"). Even my wife doesn't know that much about me!

Anyway, under "Play Store", you should see an "Installed Applications" link.

Click on that link and you will be brought to a page that shows 1) the apps currently installed on your devices 2) all the apps you have ever installed on your devices

You can quite comfortably manage your entire app library from the web browser, including install/update/delete apps. The only thing missing is probably batch install, which I hope will be added later. But if you choose multiple apps for installation, the system is smart enough to install the apps sequentially so as not to max out the device CPU and storage I/O.

Note that this only tackles app backup. App settings will only be backed up if the app explicitly built it into its code. You can check which apps backup their settings to Google's cloud by looking under "Android devices, More data stored about this device".

Not sure why this information is provided on a separate page. It would have been more intuitive if this was integrated into the "Play Store" page IMHO.

Nevertheless, this is a major improvement compared to what we had previously. All the details about the backup on each device are displayed, and managing apps from the page works very robustly from my testing (compared to Fast Web Install). All we need now is batch install function. Google, are you listening?

Read More

Jamming Flagpoles

Today I read a report about Iran's alleged "flagpole jamming network".  Reading all of this, it became plainly obvious that, "of course this is a method they're using."  Here in the States, we already use flagpoles to conceal cellular communications towers.  You can see an example of such towers here.  So, it makes a certain amount of sense that tyrants (Islamofascists, Democrats, Republicans, etc.) would use similar means to conceal a "secret" jamming network.

The one problem I have with the story reported by Fox news is simply this: a jamming system requires both power and communications connectivity.  I.e. you can't just throw a flagpole in the ground and magically make it a part of a larger jamming network. Hasn't anyone reported seemingly excessive communications or electrical work being done around these flagpoles?  Instead of offering subjective reports about juvenile cancer rates, etc., why not identify the concrete things that would prove the purpose of these mystery towers?

Media.

Read More

Sex and its irrationality

I am probably your average heterosexual male specimen. Having gone through a younger period where sex is at the top of my list of priorities, my hormones have calmed down somewhat and now it appears I can finally examine sex with more rationality.

Recently it is beginning to get to me that sex is not something that one can rationalize. It lies outside the realms of rationality. The fact that a heterosexual male gets aroused by the sight of breasts or the vagina is not something one can deconstruct. In fact, there is nothing inherently attractive about these things other than the fact that millions of years of evolution have built their sense of attractiveness into us, without which our species cannot survive. If an alien from another planet were to land on Earth, they will never understand why we find breasts or vagina sexy, much as we can understand how a peahen will find the large and bright tail of a peacock sexy.

And we have discovered for a long time that there is a large variety of sexuality outside of the standard male-female model. You have the asexual (not interested in sex), homosexual (interested in same sex), heterosexual (interested in opposite sex), bisexual (interested in both sexes), transgender (female persona trapped in a male body, and vice versa). Within a particular category, there are also huge variations i.e. interest in different sexual organs, or even symbolism of sex (eg. brassiere or panties).

It is probably unique to the human species that we have the mental capacity to tie ourselves in knots over variances in sexuality outside of the standard model. There have probably been more debates about the morality/immorality of sexual variances than how to rid our world of hunger. Heterosexuality, being the majority, has always occupied the moral high ground. Other forms of sexuality have always been the deviant form. In some countries, being homosexual or transgender can get you killed. In fact, I have encountered few people who even understand the difference between being homosexual and being transgender. Most people still associate being effeminate as being homosexual (for males), for example.

It puzzles me to no end how most people do not question the irrationality of heterosex itself, that finding sexual organs of the opposite sex attractive is just as illogical as finding sexuals organs of the same sex attractive. To most people, heterosex is just "natural", whereas other forms of sexuality are not. This is crazy, and how people can live with that conclusion is beyond me. It is probably the highest form of prejudice there ever is, that your irrationality is somehow more "natural" than some other irrationality. I mean, this is not some argument over whether one algorithm is better than another, because then you can use logic/numbers to justify your decision. This is a case of my preference is more "natural" than yours and 1) that's it, 2) end of discussion, 3) you are a deviant/outcast.

Is it just me, or does anyone else find this crazy as well?

Read More

Roomba navigation algorithm

I have been a long time user of the Roomba vacuum robot, starting from version 1 till now. It has made a big difference to my life, and I enthusiastically recommend it to all my friends.

One thing that stands Roomba apart from other vacuum robot is its navigation algorithm. There is no high-level mapping involved. Instead it uses localized decision making, much like how insects forage for food:

Our robot computes its algorithm 67 times every second, constantly stitching together information about its environment and recomputing its path. When it starts you’ll notice a spiral pattern, it’ll spiral out over a larger and larger area until it hits an object. When it finds an object, it will follow along the edge of that object for a period of time, and then it will start cris-crossing, trying to figure out the largest distance it can go without hitting another object, and that’s helping it figure out how large the space is, but if it goes for too long a period of time without hitting a wall, it’s going to start spiraling again, because it figures it’s in a wide open space, and it’s constantly calculating and figuring that out. It’s similar with the dirt sensors underneath, when one of those sensors gets tripped it changes its behaviors to cover that area. It will then go off in search of another dirty area in a straight path. The way that these different patterns pile on to each other as they go, we know that that is the most effective way to cover a room. The patterns that we chose and how the algorithm was originally developed was based off of behavior-based algorithms born out of MIT studying animals and how they go about searching areas for food. When you look at how ants and bees go out and they search areas, these kinds of coverage and figuring all of that out comes from that research. It’s not exact, obviously, I’m not saying we’re honeybees, but it’s that understanding of how to search out an area in nature that is the basis behind how our adaptive technology is developed.

The algorithm is actually quite effective and robust in the real world. For a short while, I used to house two cats in a room, and you can imagine how dusty that was. Roomba never failed to clean up the room for me after I left it to its devices. It was squeaky clean after each session, and all I had to do after that was to give the wooden floor a quick mop.

Don't take my word for it. Check out this video providing a time-lapsed coverage test of various vacuum robots, including The Roomba 5 series.

However, I never fail to notice, both online and off, how Roomba's navigation algorithm invokes all kinds of negative feeling in users. Comments typically are of the types:

... It is just going in random directions
... Personally I will not vacuum like that
... Why does it go over the same area twice?
... It is not very efficient, it takes too long 

I have seen people just standing there and supervising/criticizing their robot's every move! This is crazy! Do you stand there and supervise your washing machine? I mean, you just dump your clothes in, set a wash cycle, and come back in 45 mins or whatever. What's so difficult about that? Instead you have people just hanging around and second guessing every move the robot makes. Absurd!

I can understand maybe a robot vacuum cleaner is something new to a lot of people, it hasn't become quite as mundane as a washing machine. Or maybe a localized or bottom up algorithm is just harder to understand/accept compared to a top-down algorithm. Rodney Brooks, the co-founder of iRobot, the company that makes Roomba, is a pioneer in the field of bottom-up robotics. He famously created robotic insects that do not have a central "brain", but instead make localized decisions based on the feedback from various sensors (much like the Roomba). In the physical world, that approach turns out to be extremely robust and resilient.

AFAIK iRobot has never organized any campaign to dispel such negativity. I guess maybe it is just too difficult and technical to make people understand that a bottom-up approach is superior to a top-down (mapping) approach. I hope I can make a small difference by blogging about this here. Roomba's navigation algorithm in the real world is robust. You can move furniture around while the robot is running and not have to worry that it won't come back again. You can let it bump into your foot if you are doing stuff in the same room while it is running, but you can be assured that will not have a large impact on what it is doing.

You don't want a robot to vacuum like you do. Otherwise, cars will have legs, and washing machines will have arms. Just clear the room, set the robot down, let it run, grab a cuppa and come back an hour later. The room will be clean. Trust me.

Read More

How to make your OpenBox awesome

As a long-time Linux geek, I've discovered that there are nice aspects of GNOME/KDE, and then there's all the horrifying bits.  After almost 20 years of desktop Linux use, I've concluded that "lighter is better."  I.e. there's no real benefit to having all the bells and whistles turned on.  That's what lead me to OpenBox.

Anyone who has run OpenBox is familiar with ~/.config/openbox/autostart.sh... or at least they should be.  In case you aren't, here's the crux of it -- the autostart.sh script runs when OB starts-up, and its the vehicle through which you can start all the bits and pieces of the desktop that you might need.  Here's my autostart.sh:

#
# Openbox config for the ultimate desktop 
# 

#### policykit manager
# allows us to manage networks and other things that require root priv.
(sleep 10 && /usr/lib/policykit-1-gnome/polkit-gnome-authentication-agent-1) &

#### gnome settings daemon 
# allows us to use the gnome tools for things like display and mouse
(gnome-settings-daemon) & 

#### mount encrypted disks and set desktop wallpaper
(/home/cjs/bin/mntenc && nitrogen --set-zoom-fill /home/cjs/mine/pics/backgrounds/current.jpg ) &

#### Enable power management
# we use xfce because the gnome-power-manager went away (as a systray icon anyways)
#(gnome-power-manager) & #this went away in Debian Sid around Dec '11
(xfce4-power-manager) &

#### Thunar Daemon
# handles automounting disks, etc.
(thunar --daemon) &

#### Tint2
# our panel
(tint2) &

#### Screensaver
(xscreensaver -no-splash) &

#### Synapse
# synapse is a key-stroke app-launcher, like Gnome-DO
(synapse -s) &

#### DropBox
# sync files between multiple systems
(sleep 2s && dropbox start) &

## Launch network manager applet
# 2012/01/07 -- this has been getting launched twice, presumably by gnome-settings-daemon?
#(sleep 4s && nm-applet) &

#### volumeicon 
# allows us to control volume from an icon in the system tray
(sleep 3s && volumeicon) &

#### Clipboard manager
# parcellite is a bigger/better utility
(sleep 3s && parcellite) &

## start xsnow!
#(sleep 3s && xsnow -notrees -nokeepsnow -nosanta) &

So, in case all of this is completely Greek to you, I've included comments.  This script works provided you've got tint2, parcellite, xscreensaver, dropbox, volumeicon, and thunar installed, and in your execution path.  This script also assumes that you've got the default GNOME desktop installed for such things as settings and authentication.  The point being is that you can use this even on the default Debian 6 install, and achieve a highly functional desktop without the overhead you'd experience with a full GNOME or KDE desktop.

Read More

Shipping times from Chinese gadget sites

I like buying cheap novelty items from Chinese gadget sites. I have bought hundreds of items from various sites so far, and have had a surprisingly good experience with them overall. Incredibly I have never had an item lost thanks to AusPost.

Most items are as described and satisfactory as long as you have done your research and know what to avoid. In general, I try not to buy gadgets involving flash memory chips and batteries, because most times, you are getting inferior stuff. Lower tech items are generally OK eg. toys, phone/tablet accessories, houseware etc.

The sites I have used so far are DealExtreme, DinoDirect, Lightake and Focalprice. I am eying a few others like TinyDeal and TMart. The only problem with buying from these sites is the long shipping times. I am using this post to keep track of the delivery times of various purchases to Melbourne (Australia):

DinoDirect: Printer Ink Refill
Ordered: 9 Sep
Shipped: 13 Sep
Received: 26 Sep
Time taken:  18 days

DealExtreme: Novelty Candy Dispenser
Ordered: 2 Oct
Shipped 3 Oct
Received: 16 Oct
Time taken: 15 days

DealExtreme: Pocket Illuminated Microscope
Ordered: 1 Oct
Shipped: 3 Oct
Received: 17 Oct
Time taken: 14 days

More to come...

Read More

Yield comparison on toner cartridges/refills

Following my initial encounter with the starter toner cartridge on the Brother Hl-2132 laser printer, I have been experimenting with different toner cartridges/refills and noting the cost/yield with each method.

I bought an OEM high yield toner cartridge for $32 (excludes shipping) that claims a yield of 2600 pages. That only gave me 870 pages of actual output at about 30% coverage. The toner is also a little too light for my liking.

Then I bought a toner refill set for $47. All-in-all, I got about 600g worth of toner, even an accessory kit to convert the starter cartridge to high-yield cartridge (which I have not tried yet, but it looks easy enough). After viewing a few YouTube tutorials about the process, I set about putting 100g of toner into the empty toner cartridge.

It was surprisingly easy, much easier than filling up ink cartridges. The cap on the OEM cartridge was a snap to pop off and replace, and to my surprise, the cartridge worked flawlessly after I refilled the toner. The printout was very dark and nice, much nicer than the original toner that came with the cartridge!

And even better, the 100g of toner lasted a whopping 1200+ pages! Two subsequent refills gave me roughly the same yield at the same 30% coverage. Dividing the original cost of $47 by 5 refills (conservatively), it's like getting a new toner cartridge for $9 each time!

Update:

I discovered that for about $50, I am able to buy 1kg worth of toner over eBay, shipped from US. This is double the amount of toner for about the same price. So now I am getting a brand new high yield toner cartridge for $5 every time. It's crazy cheap! I am extremely happy with the result.

Read More

Volume normalization of MP3 audio track in AVI video files

For AVI video files with MP3 audio tracks, the best way to normalize its volume without recompression involves using FFMpeg and MP3Gain:

> ffmpeg -i input.avi -vn -acodec copy -y audio.mp3
> ffmpeg -i input.avi" -an -vcodec copy -y video.avi
> mp3gain /r audio.mp3
> ffmpeg -i audio.mp3 -i video.avi -acodec copy -vcodec copy -y output.avi

In fact, AACGain in the previous post could substitute for MP3Gain as well.

So the process basically involves extracting the audio and video tracks separately from the original video file, normalizing the audio track using MP3Gain, then muxing the audio (normalized) and video tracks back again.

Since everything is command-line driven, it will be quite straightforward to create a batch/script file that performs all 4 steps in sequence.

Read More