Off-loading Terabytes of Videos to YouTube
November 7th, 2016 | View Post


The YouTube Logo
When I first sat out to design and build kevinludlow.com, I did so under the impression that it wouldn't be long before every single thing that we had was digitally catalogued. I did so under the impression that it would become increasingly more difficult to protect oneself from having information digitally catalogued.

I'm not referring to police records and tax documents. I'm referring to that picture of you where you've had six too many to drink, your eyes are bloodshot, and there's still some puke running down your lip from when you last vomited. These are moments that we try dearly to make disappear, but they're not going to disappear. The second an iPhone or Android snaps a picture like that, not only is the content saved in an archive forever (yes, even if you delete it), but the META-data attached to the photo is frightening. There is a full story of exactly where you were, what you were doing, how long you were there for, and everything else in between included in that META-data.

I wouldn't exactly say that my attitude followed the old cliché of, "If you can't beat em, join em", but it was along those lines. The idea of my website was that if this is where we're headed, then perhaps it's just best to embrace those moments and own them as your own. So what if I had too many to drink one night. It happens. If anybody is going to own a picture of that moment, I'm going to. I'm going to be the one to make it publicly accessible for all to see. If someone should ever try to use it against me, I'm going to politely remind them that they got the image from my own personal archive. There's a power to this and I encourage everyone to embrace it.

That's the back story.

Along with that has come hundreds of hours of video footage that I've recorded over my lifetime. FINALLY, I have finished the segment of my site whereby I can archive all of my video footage not only on my own personal website, but on YouTube.

Google, like with most of their tools, has created a wonderful API for connecting with your YouTube channel and uploading videos to it. I've taken full advantage of this and in the past month or so have uploaded about 1,500 videos to their service as part of the initial experiment. Now that I finally have the tools to do it, I probably have 200-300 more hours of video that I will fully digitize and send to YouTube.

My process is simple. I start by placing all of my media content in the regular directory it falls on within my server. Think of it like a giant iPhoto gallery. It's just an enormous directory tree with about 90,000 unique memories in it (as of this writing). From there my servers run a series of automated processes that I've developed over the years. Photos are converted into different sizes and resolutions as needed by my software package. Videos are converted into OGG format (open-source video for the web) so that I can display them using HTML5 on this site. Thumbnails from the videos are automatically extracted and organized. Each entity that is created within my system is given a unique ID and catalogued within my database. There is a lookup table that will cross reference any of these unique IDs in the entire system with a specific file type (photo, video, etc.)

Focusing solely on the videos, at this point in my process there exist videos on my server that have thumbnails, that have been converted to a useful (and view-able) web format, but that lack any titles or description. I quickly scroll over the video, provide it with a title and a description, and so long as I'm okay archiving it within YouTube, I check a box and it gets marked for upload.

A standard CRON job on my server is routinely scanning the database for videos that have been approved, but not yet uploaded. Once it finds such a video, the YouTube API is invoked and the video gets uploaded to YouTube. All of the information about the video that has been populated in my database (such as the title and description) is sent over with the video for YouTube to archive. The date/time that the video goes over is logged in my database as is the unique video code that YouTube provides. From there I can reference any video in my personal catalogue by looking up the YouTube ID myself.

While I've not yet configured the videos as a YouTube plugin on the site, this is the obvious next step. Not only does it take cost off of my server, but YouTube's servers are likely significantly faster than mine.

That's the gist of how it works. I've wanted to do this for about 7 years now. I'm delighted to have finally finished the coding that makes it work.

Incidentally, if you go to my YouTube channel, you find thousands of videos, all of which originated on this site.

Newest Albums


Recent Posts
Chasing After a Hit-and-Run Suspect
Off-loading Terabytes of Videos to YouTube
Configuring ImageMagick RAW Delegates with DCRAW and UFRAW-Batch
The most amazing server uptime!
Baba O'Riley with Ghost of Paul Revere
Workshopping "Tired of Trying" with The Ghost
Acoustic Looping with a Violin Bow
DADGAD Loop with LC2 and Telecaster
APD Brought the War on Drugs to my Front Door
3rd Challenge to Dawnna Dukes - State Income Tax


Blog Archives
Recent Posts
December 2016
November 2016
November 2015
October 2015
March 2015 ( 3 )
January 2015
October 2014
September 2014 ( 4 )
August 2014 ( 6 )
July 2014 ( 7 )
June 2014 ( 8 )
May 2014 ( 3 )
April 2014 ( 2 )
February 2014
January 2014 ( 4 )
December 2013 ( 4 )
November 2013 ( 2 )
September 2013
August 2013 ( 3 )
July 2013
June 2013
May 2013 ( 2 )
March 2013 ( 2 )
February 2013
January 2013
December 2012
November 2012 ( 4 )
October 2012 ( 2 )
September 2012 ( 4 )
August 2012
July 2012 ( 8 )
June 2012
May 2012 ( 6 )
April 2012 ( 7 )
March 2012 ( 4 )
February 2012 ( 5 )
January 2012 ( 4 )
December 2011 ( 5 )
November 2011 ( 2 )
October 2011
September 2011
August 2011 ( 5 )
July 2011 ( 6 )
June 2011 ( 2 )
May 2011 ( 3 )
April 2011 ( 3 )
March 2011 ( 2 )
February 2011
January 2011
December 2010 ( 2 )
November 2010 ( 2 )
September 2010
August 2010
June 2010
March 2010
February 2010 ( 3 )
November 2009
June 2009
May 2009 ( 3 )
April 2009
March 2009 ( 2 )
February 2009
December 2008
November 2008
October 2008
May 2008 ( 2 )
March 2008
January 2008
December 2007
July 2007 ( 2 )
June 2007 ( 2 )
May 2007
December 2006
October 2006 ( 3 )
July 2006
May 2006 ( 2 )
April 2006
December 2005
October 2005
September 2005 ( 5 )
August 2005
July 2005
June 2005 ( 3 )
May 2005
April 2005
December 2004
November 2004 ( 6 )
May 2004
February 2004 ( 3 )
January 2004
December 2003 ( 9 )
November 2003 ( 5 )
August 2003
July 2003 ( 15 )
June 2003
September 2002
August 2002
May 2002
April 2002
December 2001 ( 2 )
July 2001
April 2001 ( 3 )
February 2001 ( 5 )
November 2000
September 2000
May 2000 ( 2 )
March 2000 ( 2 )
December 1999
November 1999 ( 3 )
October 1999 ( 5 )
September 1999
August 1999
July 1999 ( 8 )
June 1999 ( 2 )
May 1999 ( 3 )
April 1999
March 1999
December 1998
November 1998 ( 2 )
October 1998 ( 3 )
September 1998
July 1998 ( 2 )
June 1998
April 1998
March 1998
November 1997
October 1997 ( 2 )
March 1997
February 1997
January 1997
November 1995
September 1995 ( 2 )
Complete Listing