bash scripting: youtube downloading macro

Summary
<p>
    Once again, the command line is the root of all that is good in the world. This time, it has helped improve on a long-standing issue for me: what is the easiest way to get a copy of all the <a href='http://www.youtube.com/playlist?list=PLmku2swCXQpqWAZSscjV4h9bcLennVcif' target='_blank'>luscious melodies</a> i hear on youtube? Courtesy of <a href='http://rg3.github.io/youtube-dl/' target='_blank'>youtube-dl</a>, a nifty little command line utility, this problem has been solved. However, every once in awhile it throws errors and i wanted a wrapper bash script to take care of this and some other processing. I'll briefly go over the code.
</p>

Once again, the command line is the root of all that is good in the world. This time, it has helped improve on a long-standing issue for me: what is the easiest way to get a copy of all the luscious melodies i hear on youtube? Courtesy of youtube-dl, a nifty little command line utility, this problem has been solved. However, every once in awhile it throws errors and i wanted a wrapper bash script to take care of this and some other processing. I'll briefly go over the script (scroll to end of article to see full code).

The script solves three basic problems: re-running youtube-dl after errors, automatically deciding whether input is a list of links or a single URL, and formatting the output files. Formatting the music output files was the easiest: a base command was stored in a variable (baseCmd) that contains formatting for the title and extension (%(title)s.%(ext)s) as per the youtube-dl documentation.

Automatically choosing to run youtube-dl against a single or multiple URLs involved detecting http:// in the input and modifying the baseCmd variable from there. I avoided using getopts this time around to parse input arguments as the script was supposed to be simple as possible; i'll add this standardized input style in future revisions.

Handling errors involved taking advantage of the tee command. The main reason for using tee was to show the user output while also writing all the output from the shell to a file. After the command had finished running, successfully or not, I then grep the output, checking for ERROR, which youtube-dl outputs on error. The double passing of tee allows me to capture both STDOUT and STDERR, else just STDOUT would never capture the thrown errors and the program would exit even when problems arise. The script then calls itself passing the original argument (the base case for the recursion is the lack of an error...).

This should prove useful for people who want a little script to automate some parts of using this incredibly useful tool.

youtube.sh wrapper script
Bash
  1. # /bin/bash
  2.  
  3. # biafra ahanonu
  4. # updated: 2013.05.05
  5. # youtube-dl wrapper script to help handle errors and automate some things
  6.  
  7. printHelp(){
  8.         echo "youtube.sh FILE ARG
  9.         ARG can be:
  10.                 -h for help
  11.                 -a for list of youtube URLs in a file, use only if filename contains 'http://'
  12.                 empty for single URL or file (auto-detects)"
  13.         exit 0
  14. }
  15.  
  16. # help string
  17. helpString='-h -help --h --help'
  18. # check if first argument contains a URL
  19. URLcheck=$(echo $1 | grep 'http://' | wc -l)
  20. # base command to use
  21. baseCmd="youtube-dl -ixko %(title)s.%(ext)s --no-post-overwrites --verbose --restrict-filenames --audio-format m4a -R 30 "
  22.  
  23. # check if first argument is a help string
  24. if [[ "$helpString" == *"$1"* ]]; then
  25.         printHelp
  26. fi
  27.  
  28. # check if second arg is empty
  29. if [ -z "$2" ]; then
  30.         # If contains URL, do normally
  31.         if [ $URLcheck -eq 0 ]; then
  32.                 cmd=$baseCmd"-a $1"
  33.         # Enter batch mode if first arg is not URL
  34.         elif [ ! $URLcheck -eq 0 ]; then
  35.                 cmd=$baseCmd"$1"
  36.         fi
  37. # User should flag with -a
  38. elif [ $2 == -a ]; then
  39.         cmd=$baseCmd"-a $1"
  40. fi
  41.  
  42. echo $cmd
  43.  
  44. # continue until no error encountered
  45. error=1
  46. while [ $error -eq 1 ]; do
  47.         # clear output file
  48.         echo '' > youtube.output
  49.         # save to file and output to cmd line, captures STDOUT and STDERR to allow restart of script if errors
  50.         # $cmd | tee -ai youtube.output
  51.         $cmd > >(tee -ai youtube.output) 2> >(tee -ai youtube.output >&2)
  52.         # check if an error occurred, if so, restart script to resume download
  53.         if [ $(grep 'ERROR' youtube.output | wc -l) -eq 0 ]; then
  54.                 echo No errors found, exiting...
  55.                 error=0
  56.         else
  57.                 echo ________________________________________________________
  58.                 echo Errors found, restarting script...
  59.                 bash youtube.sh $1
  60.         fi
  61. done

-biafra
bahanonu [at] alum.mit.edu

more articles to enjoy:

bash scripting: randomly rename files
13 june 2013 | programming

Small script to enable quick randomization of files in a directory and conversion back to original names later. Original inspiration was a [...]way to blind data analysis, e.g. if studying images from an experiment and don't want to be biased by the conditions applied.

satellite-based videos: eastern europe during the russia-ukraine conflict
30 november 2022 | satellite

To visualize the nighttime lights of Eastern Europe, with a focus on times before and after the ongoing Russia-Ukraine conflict, I updated [...]my geoSatView R code originally built to view forest fires in the west coast of the United States to use satellite data from VNP46A1 and other datasets collected from the Suomi NPP VIIRS satellite.

I then created higher-quality movies in MATLAB by using the VNP46A2 Black Marble dataset collected by the same satellite, which has reduced cloud and other artifacts due to additional data processing. This allowed me to quantitate a permanent reduction in nighttime lights within Ukraine (in line with my initial hypothesis) and identify a multi-stage reduction of nighttime lights in Kiev's outer neighborhoods/metropolitan area that was greater than that seen in the city core/center. This highlights the utility of public satellite data to quickly test hypotheses and visualize large-scale changes.

I will go over how the Black Marble dataset is collected and processed along with how I created the movies and the advantages/disadvantages of each data source.

Using this platform and codebase, in follow-up posts I will look at 2021 Texas power crisis during the winter storms, vegetation changes in deforested areas or after conservation efforts, and other events.

quantized art
28 may 2012 | essay

Quantized art. The idea came about while reading how the music industry assembles top-liners, producers, artists, performers, etc. to [...]create top 40 hits. For example, there has been a recent trend in pop music to use 'drops', when the song builds to a crescendo and then a crazy, catchy bass line is released that causes everyone to dance. This has been perfected to the point where even an okay song can become popular because the producers know when to build, at what moment to intersperse catchy, meaningless lyrics and how to end the song on a high. I like the idea that art (as in paintings, drawings, etc.) can be dissected and quantified.

My first pass at developing an algorithm to break art down to its details and then use this knowledge to generate art that people would consider 'great'. We'll see how this evolves.

wary statistics #1: the tale of cdc mortality
06 april 2020 | statistics

I will briefly discuss properly interpreting data you might see in the mainstream or on social media. The takeaway: if recent data for some[...] measure (e.g. pneumonia deaths) from this year looks to be different than prior years, make sure to check that it is not an artifact of data collection or compilation.

©2006-2024 | Site created & coded by Biafra Ahanonu | Updated 21 October 2024
biafra ahanonu