110GB+ worth of photos/videos on a microSD memory card #2

In my previous post, I talked about wanting to write a script to see how far I can squeeze 110GB worth of media (photos and videos).

I wrote the script the quick-and-dirty way, in PHP. When people think of PHP, they think of web pages, but PHP can equally be used for writing your everyday scripts, just like Perl or Python.

The script resizes all photos to a maximum width/height of 1024. I found this provides maximum size reduction for the photos without sacrificing the viewing quality in full photo mode on my tablet (screen resolution: 1280x800). Obviously it will affect the quality when zooming in, but I rarely do that when flipping through the photos. Since I haven't got a tablet with retina display yet, I am not sure how that will fare, but for this exercise, I am sacrificing quality for size.

For videos, I use handbrake-cli to transcode all of them to MP4 with a maximum width of 640. Again, this reduces the size drastically, improves compatibility (since all Android tablets/smartphones I have come across so far can play MP4 with no problems) and looks OK quality-wise on my tablet.

The script takes a source directory and replicates the folder structure in a target directory. It transcodes all files with certain extensions (configurable in config.php) and places them in the corresponding target subdirectories. If the file has already been transcoded, it will not be transcoded again, so it will be relatively quick to update the target directory after adding more files to the source directory.

The script took 1.5 days to transcode my 110GB collection (bear in mind this is a older, slower AMD Turion II laptop). The result was only 29GB and fits easily into a 32GB microSD card.

I popped the microSD card into my tablet and viewed the files using QuickPic. The result was awesome.Check it out in the video below:

Note that this video was produced with the microSD card inserted into an al-cheapo single-core tablet bought for the kids during a sale. I was pleasantly surprised by how quick and smooth it was. Needless to say, on my own 10" quad-core tablet, the result was even better.

QuickPic is a better app than Gallery for this purpose because:

1. It allows you to set "Include Folders" to include only the media files from the microSD card

2. It lets you set the 'Explorer View" to browse through the folders/subfolders in a hierarchical manner.

3. It has "Stack", "Grid" and "List" views for different visual summaries of the folder content.

4. It is quick and has tons of other options.

The result of the transcoding operation exceeded all my expectations. I have a snapshot of 110GB worth of media on a tiny memory card, ready to be popped into any tablet or PC to view. At only 29GB, there's lots of room for growth, especially on a 64GB microSD card (which no doubt all tablets will support in the future, even cheapo ones).

If you think you might find this useful, you can download the script below. It includes the PHP and Handbrake-CLI binaries so it should be ready to run under Windows. It shouldn't be too difficult to adapt the script to work under other platforms as well.

After download, unzip into another directory and edit config.php. You only need to change $srcdir and $tgtdir. Then bring up a command prompt and run transcode.

Once you verify that everything works correctly, you may want to schedule it to run periodically via the Task Scheduler.

Download: Batch Media Transcoder V1.0 (ZIP 25.2MB)

Download: Batch Media Transcoder V1.0 (7-ZIP 17.1MB)

Read More

110GB+ worth of photos/videos on a microSD memory card #1

We have a family NAS where we keep all kinds of stuff. Well, actually it's an old laptop that functions as a server, with 3 x 3TB external USB HDDs attached. I run all the standard mirroring and backup on the laptop, and one of the HDD is shared over the network for everyone to dump their stuff into.

Like most family these days, we take photos and videos on a number of different devices. The smart phones, digital camera, DSLR etc. Even the kids have their own cheap digital compacts. And the devices have evolved over the years. The photos have mostly stayed JPEG, with the resolution increasing steadily. The videos, however, are a different story. Starting from the V8 (which we converted to XVID AVIs) to a Sony (which produced MPEG-1 files) to a Casio (which gave us H264/MOV files) to the current-gen Sony (MTS format), there's an incredible number of video formats we have to deal with. On top of that, we do scans of old photos and slides, the kids' artwork etc. These are all dumped into the family NAS.

To avoid being locked into a particular platform, we resisted moving to any software that relies on an underlying database for organizing the photos. Instead we created folders (2010, 2011, 2012 etc) and subfolders (eg. 20100825 Adam's Birthday Party) within the underlying filesystem to organize the photos. With the subfolders, the photos and videos are renamed to "YYYYMMDD-hhmmss" format. jhead is a great help batch-renaming of photos. For video files, the strategy varies. And not all filles can be renamed this way (eg. the scanned photos). Though crude, this has mostly worked. It is also quite flexible when it comes to backup and batch processing.

For viewing the photos, we use XnView.

IrfanView works as well, but it requires more tinkering to get it working with the different video file formats that we have. XnView works pretty much out of the box with the video files.

These two apps have a number of advantages going for them:

1. They are both freeware with lots of plugin support.

2. They work at the filesystem level, and we can easily expand/collapse the folders/subfolders to view our media archive. Some apps require you to import the media that you want to view.

3. They display both photos and videos, both at the thumbnail and individual media level. Most media viewing apps support only photos.

4. They are portable, meaning they can be easily copied/moved from machine to machine without re-installation and executed with their settings intact.

The downside is that both apps are not particularly sexy, but they do get the work done quickly and effectively.

One problem I face is that the photo/video archive on the NAS is standing at 110GB and growing steadily. I want to be able to view this archive on my Android tablet, which is a very natural medium for perusing all the photos and videos that we have accumulated. But fitting 110GB of files into the tablet is next to impossible currently (the largest microSD card is 64GB), and even if I could, the tablet won't be able to deal with all the video formats in the archive (the 1080p MTS files are particularly problematic).

So I decided to write a script to transcode all the files in the archive to a smaller size to see how much I can squeeze it down without ruining the viewing experience on the tablet. This will be the topic of my next post.

Read More

Coding the Liturgical Calendar

For a long time, I've wanted to integrate the Roman Breviary into a couple of projects (nothing public) but have never been able to find a suitable calendar, breviary or psalter implementation.  So I wrote one in python.  It requires the `dateutil` package, but other than that its standard python.  My principle use of this is going to be to create a JSON interface to get the current office, season, feast-day and reading from the 1962 Liturgical calendar.  I am working on a "modern" liturgical calendar, its just not my priority right now. 

Read More