General code improvement

Using tabs instead of spaces\nFixed argument parsing\nTidied up some code\nLong arguments are no longer supported (blame getopts).
This commit is contained in:
Xerbo 2019-07-23 15:47:48 +01:00
parent 6d745c1228
commit 3821157d09
2 changed files with 108 additions and 111 deletions

View file

@ -12,22 +12,22 @@ Windows users can get it to work via Microsoft's [WSL](https://docs.microsoft.co
## Usage ## Usage
Make it executable with Make it executable with
`chmod +x faraffinity-dl-ng` `chmod +x faraffinity-dl`
And run it with And then run it with
`./furaffinity-dl-ng section/username` `./furaffinity-dl section/username`
All files from the given section and user will be downloaded to the current directory. All files from the given section and user will be downloaded to the current directory.
### Examples ### Examples
`./furaffinity-dl-ng gallery/mylafox` `./furaffinity-dl gallery/mylafox`
`./furaffinity-dl-ng -o=mylasArt gallery/mylafox` `./furaffinity-dl -o mylasArt gallery/mylafox`
`./furaffinity-dl-ng --out=koulsFavs favorites/koul` `./furaffinity-dl -o koulsFavs favorites/koul`
You can also log in to download restricted content. To do that, log in to FurAffinity in your web browser, export cookies to a file from your web browser in Netscape format (there are extensions to do that [for Firefox](https://addons.mozilla.org/en-US/firefox/addon/ganbo/) and [for Chrome/Vivaldi](https://chrome.google.com/webstore/detail/cookiestxt/njabckikapfpffapmjgojcnbfjonfjfg)) and pass them to the script as a second parameter, like this: You can also log in to download restricted content. To do that, log in to FurAffinity in your web browser, export cookies to a file from your web browser in Netscape format (there are extensions to do that [for Firefox](https://addons.mozilla.org/en-US/firefox/addon/ganbo/) and [for Chrome/Vivaldi](https://chrome.google.com/webstore/detail/cookiestxt/njabckikapfpffapmjgojcnbfjonfjfg)) and pass them to the script as a second parameter, like this:
`./furaffinity-dl -c=/path/to/your/cookies.txt gallery/gonnaneedabiggerboat` `./furaffinity-dl -c /path/to/your/cookies.txt gallery/gonnaneedabiggerboat`
## TODO ## TODO
* Download user bio, post tags and ideally user comments * Download user bio, post tags and ideally user comments

View file

@ -1,40 +1,39 @@
#!/bin/bash #!/bin/bash
set -e set -e
# Detect installed applications # Detect installed metadata injectors
if [ -f /usr/bin/eyeD3 ]; then
eyed3=true eyed3=true
else if [ -z $(command -v eyeD3) ]; then
eyed3=false eyed3=false
echo "INFO: eyed3 is not installed, no metadata will be injected into music files." echo "INFO: eyed3 is not installed, no metadata will be injected into music files."
fi fi
if [ -f /usr/bin/exiftool ]; then
exiftool=true exiftool=true
else if [ -z $(command -v exiftool) ]; then
exiftool=false exiftool=false
echo "INFO: exiftool is not installed, no metadata will be injected into pictures." echo "INFO: exiftool is not installed, no metadata will be injected into pictures."
fi fi
# Helper functions # Helper functions
help() { help() {
echo "Usage: $0 SECTION/USER [ARGUMENTS] echo "Usage: $0 [ARGUMENTS] SECTION/USER
Downloads the entire gallery/scraps/favorites of any furaffinity user. Downloads the entire gallery/scraps/favorites of any furaffinity user.
Arguments: Arguments:
-h --help This text -h (H)elp screen
-i --http Use an insecure connection -i Use an (I)nsecure connection when downloading
-o --out The directory to put files in -o The (O)utput directory to put files in
-c --cookiefile If you need to download restricted content -c If you need to download restricted content
you can provide a path to a cookie file you can provide a path to a (C)ookie file
-p --nometa Plain file without any additional metadata -p (P)lain file without any additional metadata
Examples: Examples:
$0 gallery/mylafox $0 gallery/mylafox
$0 -o=myla gallery/mylafox $0 -o mylasArt gallery/mylafox
$0 --out=koul favorites/koul $0 -o koulsFavs favorites/koul
You can also log in to FurAffinity to download restricted content, like this: You can also log in to FurAffinity to download restricted content, like this:
$0 -c=/path/to/your/cookies.txt gallery/gonnaneedabiggerboat $0 -c /path/to/your/cookies.txt gallery/gonnaneedabiggerboat
DISCLAIMER: It is your own responsibility to check whether batch downloading is allowed by FurAffinity terms of service and to abide by them." DISCLAIMER: It is your own responsibility to check whether batch downloading is allowed by FurAffinity terms of service and to abide by them."
exit 1 exit 1
@ -44,46 +43,42 @@ cleanup() {
} }
# Arguments # Arguments
if [ $# -eq 0 ]; then [[ $# -eq 0 ]] && help
help
fi
prefix="https:"
outdir="." outdir="."
metadata=true; prefix="https:"
case "$1" in metadata=true
# Help while getopts 'o:c:i:p:h:' flag; do
-h|--help) help;; case "${flag}" in
o) outdir=${OPTARG};;
# HTTP / HTTPS c) cookiefile=${OPTARG};;
-i|--http) prefix="http:";; i) prefix="http:";;
p) metadata=false;;
# Output directory h) help;;
-o=*|--out=*) outdir="${1#*=}"; shift 1;; *) help;;
# Cookie file
-c=*|--cookiefile=*) cookiefile="${1#*=}"; shift 1;;
# Metadata
-p|--nometa) meta=false;;
esac esac
done
# Attempt to create the output directory
mkdir -p "$outdir" mkdir -p "$outdir"
runtime_dir="$HOME"'/.cache/furaffinity-dl-ng' # Setup runtime directory
runtime_dir="$HOME"'/.cache/furaffinity-dl'
mkdir -p "$runtime_dir" mkdir -p "$runtime_dir"
tempfile="$(umask u=rwx,g=,o= && mktemp $runtime_dir/fa-dl.XXXXXXXXXX)" tempfile="$(umask u=rwx,g=,o= && mktemp $runtime_dir/fa-dl.XXXXXXXXXX)"
# Call cleanup function on exit
trap cleanup EXIT trap cleanup EXIT
if [ "$cookiefile" = "" ]; then if [ -z "$cookiefile" ]; then
# Set wget with a custom user agent # Set wget with a custom user agent
fwget() { fwget() {
wget -nv --user-agent="Mozilla/5.0 furaffinity-dl-ng (https://github.com/Xerbo/furaffinity-dl-ng)" $* wget -nv --user-agent="Mozilla/5.0 furaffinity-dl (https://github.com/Xerbo/furaffinity-dl)" $*
} }
else else
# Set wget with a custom user agent and cookies # Set wget with a custom user agent and cookies
fwget() { fwget() {
wget -nv --user-agent="Mozilla/5.0 furaffinity-dl-ng (https://github.com/Xerbo/furaffinity-dl-ng)" --load-cookies "$cookiefile" $* wget -nv --user-agent="Mozilla/5.0 furaffinity-dl (https://github.com/Xerbo/furaffinity-dl)" --load-cookies "$cookiefile" $*
} }
fi fi
@ -92,7 +87,7 @@ url="https://www.furaffinity.net/${@: -1}"
# Iterate over the gallery pages with thumbnails and links to artwork view pages # Iterate over the gallery pages with thumbnails and links to artwork view pages
while true; do while true; do
fwget "$url" -O "$tempfile" fwget "$url" -O "$tempfile"
if [ "$cookiefile" != "" ] && grep -q 'furaffinity.net/login/' "$tempfile"; then if [ -n "$cookiefile" ] && grep -q 'furaffinity.net/login/' "$tempfile"; then
echo "ERROR: You have provided a cookies file, but it does not contain valid cookies. echo "ERROR: You have provided a cookies file, but it does not contain valid cookies.
If this file used to work, this means that the cookies have expired; If this file used to work, this means that the cookies have expired;
@ -103,7 +98,7 @@ in Netscape format (this is normally done through \"cookie export\" browser exte
and supplied the correct path to the cookies.txt file to this script. and supplied the correct path to the cookies.txt file to this script.
If that doesn't resolve the issue, please report the problem at If that doesn't resolve the issue, please report the problem at
https://github.com/Xerbo/furaffinity-dl-ng/issues" >&2 https://github.com/Xerbo/furaffinity-dl/issues" >&2
exit 1 exit 1
fi fi
@ -129,30 +124,32 @@ https://github.com/Xerbo/furaffinity-dl-ng/issues" >&2
# Get metadata # Get metadata
description=$(grep 'og:description" content="' "$tempfile" | cut -d '"' -f4) description=$(grep 'og:description" content="' "$tempfile" | cut -d '"' -f4)
title=$(grep 'og:title" content="' "$tempfile" | cut -d '"' -f4) title=$(grep 'og:title" content="' "$tempfile" | cut -d '"' -f4)
title="${title%" by"*}" # Remove the " by Artist" bit
file_type=${image_url##*.} file_type=${image_url##*.}
file="$outdir/$title.$file_type" file="$outdir/$title.$file_type"
# Download the image
wget "$image_url" -O "$file" wget "$image_url" -O "$file"
# Add metadata # Add metadata
if [ $file_type == "mp3" ] || [ $file_type == "wav" ] || [ $file_type == "wmv" ] || [ $file_type == "ogg" ] || [ $file_type == "flac" ]; then if [ $file_type == "mp3" ] || [ $file_type == "wav" ] || [ $file_type == "wmv" ] || [ $file_type == "ogg" ] || [ $file_type == "flac" ]; then
# Use eyeD3 for injecting injecting metadata into audio files (if it's installed) # Use eyeD3 for injecting metadata into audio files (if it's installed)
if [ $eyed3 ] && [ $metadata ]; then if [ $eyed3 ] && [ $metadata ]; then
if [ -z "$description" ]; then if [ -z "$description" ]; then
eyeD3 -t "$title" "$file" eyeD3 -t "$title" "$file"
else else
# HACK eyeD3 throws an error if a description containing a ":" # HACK: eyeD3 throws an error if a description containing a ":"
eyeD3 -t "$title" --add-comment "${description//:/\\:}" "$file" eyeD3 -t "$title" --add-comment "${description//:/\\:}" "$file"
fi fi
fi fi
elif [ $file_type == "png" ] || [ $file_type == "jpg" ] || [ $file_type == "jpeg" ]; then elif [ $file_type == "png" ] || [ $file_type == "jpg" ] || [ $file_type == "jpeg" ]; then
# Use exiftool for injecting metadata into pictures (if it's installed) # Use exiftool for injecting metadata into pictures (if it's installed)
if [ $exiftool ] && [ $metadata ]; then if [ $exiftool ] && [ $metadata ]; then
exiftool "$file" -description="$description" -title="$title" exiftool "$file" -description="$description" -title="$title" -overwrite_original
fi fi
fi fi
done done
[ "$next_page_url" = "" ] && break [ -z "$next_page_url" ] && break
url='https://www.furaffinity.net'"$next_page_url" url='https://www.furaffinity.net'"$next_page_url"
done done