so-rose
/
convmlv
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 3 Wiki Activity

Fixed the huge colorspace bug (in a weird way - see Nasty Hacks in source). Fixed sRange 1 n error, where no output would be given. Implemented hqdn3d denoiser, and ffmpeg deshake, filters. 

Expanded filter framework; list now formats correctly.
modularize
Sofus Albert Høgsbro Rose 2016-06-27 00:59:14 -04:00
parent 784437b9e1
commit b4e83a6e2f
7 changed files with 230 additions and 178 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
7D_badpixels.txt 100644 → 100755
View File

@ -9,6 +9,10 @@
# DOA - Discovered 3/14/16 # DOA - Discovered 3/14/16
432 481 0 432 481 0
432 600 0
432 601 0
433 601 0
544 109 0
197 1013 0 197 1013 0
167 677 0 167 677 0
343 803 0 343 803 0

5
CHANGELOG
View File

@ -1,4 +1,9 @@
[code] [code]
[b]v1.9.2[/b]: [url=https://bitbucket.org/so-rose/convmlv/src]https://bitbucket.org/so-rose/convmlv/src[/url]
[size=8pt]*Atadenoise Support! It's a great denoising method, used with "-N <A>-<B>" See the documentation for how to configure A and B values.[/size]
[size=8pt]*General FFMPEG Filter Support: The framework for any ffmpeg -vf filters exists now! lut3d and atadenoise are currently implemented.[/size]
[size=8pt]*Bug Fixes: LUTs can now be applied to EXR's! Other bugs fixed too.[/size]
[b]v1.9.1[/b]: [url=https://bitbucket.org/so-rose/convmlv/src]https://bitbucket.org/so-rose/convmlv/src[/url] [b]v1.9.1[/b]: [url=https://bitbucket.org/so-rose/convmlv/src]https://bitbucket.org/so-rose/convmlv/src[/url]
[size=8pt]*Fully Supported Config Files! Check the help page; all options for which this makes sense can be specified in global, local, or file-specific config files.[/size] [size=8pt]*Fully Supported Config Files! Check the help page; all options for which this makes sense can be specified in global, local, or file-specific config files.[/size]
[size=8pt]*File-Specific Blocks: With this, just make one config file w/any special options per file, and you're done. Huge time saver in production situations![/size] [size=8pt]*File-Specific Blocks: With this, just make one config file w/any special options per file, and you're done. Huge time saver in production situations![/size]

136
README.md
View File

@ -19,7 +19,9 @@ INFO:
--> Acceptable Inputs: MLV, RAW, DNG Folder. --> Acceptable Inputs: MLV, RAW, DNG Folder.
--> Option Input: From command line or config file. --> Option Input: From command line or config file.
VERSION: 1.9.2 --> Forum Post: http://www.magiclantern.fm/forum/index.php?topic=16799.
VERSION: 1.9.3
MANUAL DEPENDENCIES: MANUAL DEPENDENCIES:
-mlv_dump: Required. http://www.magiclantern.fm/forum/index.php?topic=7122.0 -mlv_dump: Required. http://www.magiclantern.fm/forum/index.php?topic=7122.0
@ -99,6 +101,14 @@ OPTIONS, RAW DEVELOPMENT:
--> A: 0 to 0.3. B: 0 to 5. A reacts to abrupt noise (splotches), B reacts to noise over time (fast motion causes artifacts). --> A: 0 to 0.3. B: 0 to 5. A reacts to abrupt noise (splotches), B reacts to noise over time (fast motion causes artifacts).
--> Subtle: 0.03-0.04. High: 0.15-0.04. High, Predictable Motion: 0.15-0.07 --> Subtle: 0.03-0.04. High: 0.15-0.04. High, Predictable Motion: 0.15-0.07
-Q [i-i:i-i] HQ_NOISE - Apply 3D denoising filter.
--> In depth explanation: https://mattgadient.com/2013/06/29/in-depth-look-at-de-noising-in-handbrake-with-imagevideo-examples/ .
--> Spacial/Temporal (S/T). S will soften/blur/smooth, T will remove noise without doing that but may create artifacts.
--> Luma/Chroma (L/C). L is the detail, C is the color. Each one's denoising may be manipulated Spacially or Temporally.
--> Options: <LS>-<CS>:<LT>-<CT>
--> Weak: 2-1:2-3. Medium: 3-2:2-3. Strong: 7-7:5-5
-g [0:4] SPACE - Output color transformation. -g [0:4] SPACE - Output color transformation.
--> 0: Linear. 1: 2.2 (Adobe RGB). 2: 1.8 (ProPhoto RGB). 3: sRGB. 4: BT.709. --> 0: Linear. 1: 2.2 (Adobe RGB). 2: 1.8 (ProPhoto RGB). 3: sRGB. 4: BT.709.
@ -112,7 +122,7 @@ OPTIONS, COLOR:
-l <path> LUT - Specify a LUT to apply. -l <path> LUT - Specify a LUT to apply.
--> Supports cube, 3dl, dat, m3d. --> Supports cube, 3dl, dat, m3d.
-S [int] SATPOINT - Specify the 14-bit saturation point of your camera. -S [int] SATPOINT - Specify the 14-bit saturation point of your camera. You don't usually want to.
--> Lower if -H1 yields purple highlights. Must be correct for highlight reconstruction. --> Lower if -H1 yields purple highlights. Must be correct for highlight reconstruction.
--> Determine using the max value of 'dcraw -D -j -4 -T' --> Determine using the max value of 'dcraw -D -j -4 -T'
@ -122,6 +132,9 @@ OPTIONS, COLOR:
OPTIONS, FEATURES: OPTIONS, FEATURES:
-D DESHAKE - Stabilize the video using the wonderful ffmpeg "deshake" module.
--> You'll probably wish to crop/scale the output in editing, to avoid edge artifacts.
-u DUAL_ISO - Process as dual ISO. -u DUAL_ISO - Process as dual ISO.
-b BADPIXELS - Fix focus pixels issue using dfort's script. -b BADPIXELS - Fix focus pixels issue using dfort's script.
@ -134,7 +147,8 @@ OPTIONS, FEATURES:
--> If the file extension is '.darkframe', the file will be used as the preaveraged dark frame. --> If the file extension is '.darkframe', the file will be used as the preaveraged dark frame.
-R <path> dark_out - Specify to create a .darkframe file from passed in MLV. -R <path> dark_out - Specify to create a .darkframe file from passed in MLV.
--> Outputs <arg>.darkframe file to <path>. --> Usage: 'convmlv -R <path> <input>.MLV'
--> Averages <input>.MLV to create <path>.darkframe.
--> THE .darkframe EXTENSION IS ADDED FOR YOU. --> THE .darkframe EXTENSION IS ADDED FOR YOU.
@ -151,113 +165,45 @@ OPTIONS, INFO:
--> There's no automatic way to install these. See http://www.magiclantern.fm/forum/index.php?topic=16799.0 . --> There's no automatic way to install these. See http://www.magiclantern.fm/forum/index.php?topic=16799.0 .
CONFIG FILE: CONFIG FILE:
You do not need to type in all the arguments each time: Config files, another way to specify options, can save you time & lend Config files, another way to specify options, can save you time & lend you convenience in production situations.
you convenience in production situations.
GLOBAL: /home/sofus/convmlv.conf GLOBAL: /home/sofus/convmlv.conf
LOCAL: Specify -C/--config. LOCAL: Specify -C/--config.
Some options have an uppercased VARNAME, ex. OUTDIR. In a convmlv config file, you can specify this option
in the following format, line by line: SYNTAX:
Most options listed above have an uppercased VARNAME, ex. OUTDIR. Yu can specify such options in config files, as such:
<VARNAME> <VALUE> <VARNAME> <VALUE>
If the value is a true/false flag, simply specifying VARNAME is enough. Otherwise, normal rules for the value applies. One option per line only. Indentation by tabs or spaces is allowed, but not enforced.
Options override each other as such: Comments Lines starting with # are comments.
-LOCAL overrides GLOBAL config.
-Passed arguments override both configs. You may name a config using:
-Lines starting with # are comments.
-Name a config using the VARNAME: CONFIG_NAME <name> CONFIG_NAME <name>
Flags If the value is a true/false flag (ex. IMAGE), simply specifying VARNAME is enough. THere is no VALUE.
OPTION ORDER OF PRECEDENCE Options override each other as such:
-LOCAL options overwrite GLOBAL options.
-COMMAND LINE options overwrite LOCAL & GLOBAL options.
-FILE SPECIFIC options overwrite ALL ABOVE options.
File-Specific Block: A LOCAL config file lets you specify options for specific input names: File-Specific Block: A LOCAL config file lets you specify options for specific input names:
/ <TRUNCATED INPUTNAME> / <TRUNCATED INPUTNAME>
...specify options ...options here will only be
* *
File-Specific Blocks override all other options. This allows one to create You must use the truncated (no .mlv or .raw) input name after the /. Nested blocks will fail.
a single config file to batch-develop several input files/folders at once, after deciding how each one should
look/be configured individually.
Notes on Usage: With a single config file, you can control the development options of multiple inputs as specifically and/or generically
-You must use the truncated (no .mlv or .raw) input name after the /. as you want. Batch developing everything can then be done with a single, powerful commmand.
-No nested blocks.
-Indentation by tabs or spaces is allowed, but not enforced. -w [0:2] WHITE - This is a modal white balance setting.
--> 0: Auto WB. 1: Camera WB (default). 2: No Change.
-l <path> LUT - Specify a LUT to apply.
--> Supports cube, 3dl, dat, m3d.
--> LUT cannot be applied to EXR sequences.
-S [int] SATPOINT - Specify the 14-bit saturation point of your camera.
--> Lower if -H1 yields purple highlights. Must be correct for highlight reconstruction.
--> Determine using the max value of 'dcraw -D -j -4 -T'
--white-speed [int] WHITE_SPD - Samples used to calculate AWB
--allow-white-clip WHITE_CLIP - Let White Balance multipliers clip.
OPTIONS, FEATURES:
-u DUAL_ISO - Process as dual ISO.
-b BADPIXELS - Fix focus pixels issue using dfort's script. Contact me with any feedback or questions at convmlv@sofusrose.com, or PM me (so-rose) on the ML forums!
-a <path> BADPIXEL_PATH - Use your own .badpixels file.
--> How to: http://www.dl-c.com/board/viewtopic.php?f=4&t=686
-F <path> DARKFRAME - This is the path to a "dark frame MLV"; effective for noise reduction.
--> How to: Record 5 sec w/lens cap on & same settings as footage. Pass MLV in here.
--> If the file extension is '.darkframe', the file will be used as the preaveraged dark frame.
-R <path> dark_out - Specify to create a .darkframe file from passed in MLV.
--> Outputs <arg>.darkframe file to <path>.
--> THE .darkframe EXTENSION IS ADDED FOR YOU.
OPTIONS, INFO:
-q Output MLV settings.
-K Debian Package Deps - Output package dependecies.
--> Install (Debian only): sudo apt-get install $ (./convmlv -K)
-Y Python Deps - Lists Python dependencies. Works directly with pip.
-->Install (Linux): sudo pip3 install $ (./convmlv -Y)
-N Manual Deps - Lists manual dependencies, which must be downloaded by hand.
--> There's no automatic way to install these. See http://www.magiclantern.fm/forum/index.php?topic=16799.0 .
CONFIG FILE:
You do not need to type in all the arguments each time: Config files, another way to specify options, can save you time & lend
you convenience in production situations.
GLOBAL: /home/sofus/convmlv.conf
LOCAL: Specify -C/--config.
Some options have an uppercased VARNAME, ex. OUTDIR. In a convmlv config file, you can specify this option
in the following format, line by line:
<VARNAME> <VALUE>
If the value is a true/false flag, simply specifying VARNAME is enough. Otherwise, normal rules for the value applies.
Options override each other as such:
-LOCAL overrides GLOBAL config.
-Passed arguments override both configs.
-Lines starting with # are comments.
-Name a config using the VARNAME: CONFIG_NAME <name>
File-Specific Block: A LOCAL config file lets you specify options for specific input names:
/ <TRUNCATED INPUTNAME>
...specify options
*
File-Specific Blocks override all other options. This allows one to create
a single config file to batch-develop several input files/folders at once, after deciding how each one should
look/be configured individually.
Notes on Usage:
-You must use the truncated (no .mlv or .raw) input name after the /.
-No nested blocks.
-Indentation by tabs or spaces is allowed, but not enforced.
``` ```

9
configs/atadenoise-test.conf
View File

@ -20,22 +20,22 @@ PYTHON python3
IMAGE IMAGE
IMG_FMT 0 IMG_FMT 0
#~ MOVIE MOVIE
PROXY 0 PROXY 0
#~ PROXY_SCALE 50% #~ PROXY_SCALE 50%
KEEP_DNGS KEEP_DNGS
FRAME_RANGE 300-350 FRAME_RANGE 300-310
#UNCOMP UNCOMP
### <------------------ RAW OPTIONS ------------------> ### <------------------ RAW OPTIONS ------------------>
DEMO_MODE 2 DEMO_MODE 2
HIGHLIGHT_MODE 0 HIGHLIGHT_MODE 0
#~ TEMP_NOISE 0.15-0.04 #~ TEMP_NOISE 0.15-0.04
#~ HQ_NOISE 7-7:5-5
SPACE 0 SPACE 0
@ -44,6 +44,7 @@ SPACE 0
WHITE 1 WHITE 1
### <------------------ FEATURE OPTIONS ------------------> ### <------------------ FEATURE OPTIONS ------------------>
DESHAKE
#DARKFRAME ./df.darkframe #DARKFRAME ./df.darkframe
#~ LUT /home/sofus/subhome/src/convmlv/test_footage/girl_lut_test.cube #~ LUT /home/sofus/subhome/src/convmlv/test_footage/girl_lut_test.cube

2
configs/template.txt
View File

@ -40,6 +40,7 @@ HIGHLIGHT_MODE 0
CHROMA_SMOOTH 1 CHROMA_SMOOTH 1
WAVE_NOISE 50 WAVE_NOISE 50
TEMP_NOISE 0.03-0.04 TEMP_NOISE 0.03-0.04
HQ_NOISE 2-1:2-3
SPACE 3 SPACE 3
#SHALLOW #SHALLOW
@ -56,6 +57,7 @@ WHITE 0
#WHITE_CLIP #WHITE_CLIP
### <------------------ FEATURE OPTIONS ------------------> ### <------------------ FEATURE OPTIONS ------------------>
#DESHAKE
#DUAL_ISO #DUAL_ISO
BADPIXELS BADPIXELS

207
convmlv.sh
View File

@ -3,13 +3,14 @@
#TODO: #TODO:
#~ Stats for .RAW files. #~ Stats for .RAW files.
#~ Integrate anti-vertical banding. May require being able to use multiple darkframe files. #~ Integrate anti-vertical banding. May require being able to use multiple darkframe files.
#~ Atadenoise integration.
#~ Better Preview: #~ Better Preview:
#~ --> A different module (like -e) for live viewing of footage, under convmlv settings. Danne is working on this :). #~ --> A different module (like -e) for live viewing of footage, under convmlv settings. Danne is working on this :).
#BUG: Relative OUTDIR makes baxpixel generation fail if ./mlv2badpixels.sh doesn't exist. Fixed on Linux only. #BUG: Relative OUTDIR makes baxpixel generation fail if ./mlv2badpixels.sh doesn't exist. Fixed on Linux only.
#BUG: Colorspace Bug Affecting all but EXR output. #CONCERN: Weirdness with color spaces in general. No impact to user; just some weirdly placed sRGB conversions if'ed by DPX.
#~ --> See Nasty Hacks Part 1 - 3. FML :O.
#~ I'll feel better about this after quite a while
#~ The MIT License (MIT) #~ The MIT License (MIT)
@ -35,7 +36,7 @@
#~ SOFTWARE. #~ SOFTWARE.
#BASIC VARS #BASIC VARS
VERSION="1.9.2" #Version string. VERSION="1.9.3" #Version string.
INPUT_ARGS=$(echo "$@") #The original input argument string. INPUT_ARGS=$(echo "$@") #The original input argument string.
if [[ $OSTYPE == "linux-gnu" ]]; then if [[ $OSTYPE == "linux-gnu" ]]; then
@ -130,9 +131,13 @@ setDefaults() { #Set all the default variables. Run here, and also after each AR
#FFMPEG Filters #FFMPEG Filters
FFMPEG_FILTERS=false #Whether or not FFMPEG filters are going to be used. FFMPEG_FILTERS=false #Whether or not FFMPEG filters are going to be used.
TEMP_NOISE="" #Temporal noise reduction. TEMP_NOISE="" #Temporal noise reduction.
isTemp=false tempDesc=""
LUT="" LUT=""
isLUT=false lutDesc=""
DESHAKE=""
deshakeDesc=""
HQ_NOISE="" #hqdn3d noise reduction.
hqDesc=""
} }
setDefaults #Run now, but also later. setDefaults #Run now, but also later.
@ -148,6 +153,8 @@ INFO:
--> Acceptable Inputs: MLV, RAW, DNG Folder. --> Acceptable Inputs: MLV, RAW, DNG Folder.
--> Option Input: From command line or config file. --> Option Input: From command line or config file.
--> Forum Post: http://www.magiclantern.fm/forum/index.php?topic=16799.
$(echo -e "VERSION: ${VERSION}") $(echo -e "VERSION: ${VERSION}")
MANUAL DEPENDENCIES: MANUAL DEPENDENCIES:
@ -228,6 +235,14 @@ OPTIONS, RAW DEVELOPMENT:
--> A: 0 to 0.3. B: 0 to 5. A reacts to abrupt noise (splotches), B reacts to noise over time (fast motion causes artifacts). --> A: 0 to 0.3. B: 0 to 5. A reacts to abrupt noise (splotches), B reacts to noise over time (fast motion causes artifacts).
--> Subtle: 0.03-0.04. High: 0.15-0.04. High, Predictable Motion: 0.15-0.07 --> Subtle: 0.03-0.04. High: 0.15-0.04. High, Predictable Motion: 0.15-0.07
-Q [i-i:i-i] HQ_NOISE - Apply 3D denoising filter.
--> In depth explanation: https://mattgadient.com/2013/06/29/in-depth-look-at-de-noising-in-handbrake-with-imagevideo-examples/ .
--> Spacial/Temporal (S/T). S will soften/blur/smooth, T will remove noise without doing that but may create artifacts.
--> Luma/Chroma (L/C). L is the detail, C is the color. Each one's denoising may be manipulated Spacially or Temporally.
--> Options: <LS>-<CS>:<LT>-<CT>
--> Weak: 2-1:2-3. Medium: 3-2:2-3. Strong: 7-7:5-5
-g [0:4] SPACE - Output color transformation. -g [0:4] SPACE - Output color transformation.
--> 0: Linear. 1: 2.2 (Adobe RGB). 2: 1.8 (ProPhoto RGB). 3: sRGB. 4: BT.709. --> 0: Linear. 1: 2.2 (Adobe RGB). 2: 1.8 (ProPhoto RGB). 3: sRGB. 4: BT.709.
@ -241,7 +256,7 @@ OPTIONS, COLOR:
-l <path> LUT - Specify a LUT to apply. -l <path> LUT - Specify a LUT to apply.
--> Supports cube, 3dl, dat, m3d. --> Supports cube, 3dl, dat, m3d.
-S [int] SATPOINT - Specify the 14-bit saturation point of your camera. -S [int] SATPOINT - Specify the 14-bit saturation point of your camera. You don't usually want to.
--> Lower if -H1 yields purple highlights. Must be correct for highlight reconstruction. --> Lower if -H1 yields purple highlights. Must be correct for highlight reconstruction.
--> Determine using the max value of 'dcraw -D -j -4 -T' --> Determine using the max value of 'dcraw -D -j -4 -T'
@ -251,6 +266,9 @@ OPTIONS, COLOR:
OPTIONS, FEATURES: OPTIONS, FEATURES:
-D DESHAKE - Stabilize the video using the wonderful ffmpeg "deshake" module.
--> You'll probably wish to crop/scale the output in editing, to avoid edge artifacts.
-u DUAL_ISO - Process as dual ISO. -u DUAL_ISO - Process as dual ISO.
-b BADPIXELS - Fix focus pixels issue using dfort's script. -b BADPIXELS - Fix focus pixels issue using dfort's script.
@ -263,7 +281,8 @@ OPTIONS, FEATURES:
--> If the file extension is '.darkframe', the file will be used as the preaveraged dark frame. --> If the file extension is '.darkframe', the file will be used as the preaveraged dark frame.
-R <path> dark_out - Specify to create a .darkframe file from passed in MLV. -R <path> dark_out - Specify to create a .darkframe file from passed in MLV.
--> Outputs <arg>.darkframe file to <path>. --> Usage: 'convmlv -R <path> <input>.MLV'
--> Averages <input>.MLV to create <path>.darkframe.
--> THE .darkframe EXTENSION IS ADDED FOR YOU. --> THE .darkframe EXTENSION IS ADDED FOR YOU.
@ -280,38 +299,47 @@ OPTIONS, INFO:
--> There's no automatic way to install these. See http://www.magiclantern.fm/forum/index.php?topic=16799.0 . --> There's no automatic way to install these. See http://www.magiclantern.fm/forum/index.php?topic=16799.0 .
CONFIG FILE: CONFIG FILE:
You do not need to type in all the arguments each time: Config files, another way to specify options, can save you time & lend Config files, another way to specify options, can save you time & lend you convenience in production situations.
you convenience in production situations.
GLOBAL: $HOME/convmlv.conf $(echo -e "\e[1mGLOBAL\e[0m"): $HOME/convmlv.conf
LOCAL: Specify -C/--config. $(echo -e "\e[1mLOCAL\e[0m"): Specify -C/--config.
$(echo -e "\e[1mSYNTAX:\e[0m")
Most options listed above have an uppercased VARNAME, ex. OUTDIR. Yu can specify such options in config files, as such:
Some options have an uppercased VARNAME, ex. OUTDIR. In a convmlv config file, you can specify this option
in the following format, line by line:
<VARNAME> <VALUE> <VARNAME> <VALUE>
If the value is a true/false flag, simply specifying VARNAME is enough. Otherwise, normal rules for the value applies. One option per line only. Indentation by tabs or spaces is allowed, but not enforced.
Options override each other as such: $(echo -e "\e[1mComments\e[0m") Lines starting with # are comments.
-LOCAL overrides GLOBAL config.
-Passed arguments override both configs. You may name a config using:
-Lines starting with # are comments.
-Name a config using the VARNAME: CONFIG_NAME <name> CONFIG_NAME <name>
$(echo -e "\e[1mFlags\e[0m") If the value is a true/false flag (ex. IMAGE), simply specifying VARNAME is enough. THere is no VALUE.
$(echo -e "\e[1mOPTION ORDER OF PRECEDENCE\e[0m") Options override each other as such:
-LOCAL options overwrite GLOBAL options.
-COMMAND LINE options overwrite LOCAL & GLOBAL options.
-FILE SPECIFIC options overwrite ALL ABOVE options.
File-Specific Block: A LOCAL config file lets you specify options for specific input names: $(echo -e "\e[1mFile-Specific Block\e[0m"): A LOCAL config file lets you specify options for specific input names:
/ <TRUNCATED INPUTNAME> / <TRUNCATED INPUTNAME>
...specify options ...options here will only be
* *
$(echo -e "\e[1mFile-Specific Blocks override all other options.\e[0m") This allows one to create You must use the truncated (no .mlv or .raw) input name after the /. Nested blocks will fail.
a single config file to batch-develop several input files/folders at once, after deciding how each one should
look/be configured individually.
Notes on Usage: With a single config file, you can control the development options of multiple inputs as specifically and/or generically
-You must use the truncated (no .mlv or .raw) input name after the /. as you want. Batch developing everything can then be done with a single, powerful commmand.
-No nested blocks.
-Indentation by tabs or spaces is allowed, but not enforced.
Contact me with any feedback or questions at convmlv@sofusrose.com, or PM me (so-rose) on the ML forums!
EOF EOF
} }
@ -456,7 +484,22 @@ evalConf() {
bVal=$(echo "${vals}" | cut -d"-" -f2) bVal=$(echo "${vals}" | cut -d"-" -f2)
TEMP_NOISE="atadenoise=0a=${aVal}:0b=${bVal}:1a=${aVal}:1b=${bVal}:2a=${aVal}:2b=${bVal}" TEMP_NOISE="atadenoise=0a=${aVal}:0b=${bVal}:1a=${aVal}:1b=${bVal}:2a=${aVal}:2b=${bVal}"
isTemp=true tempDesc="Temporal Denoiser"
FFMPEG_FILTERS=true
;;
"HQ_NOISE")
vals="$(echo "${line}" | cut -d$' ' -f2)"
S=`echo "${vals}" | cut -d$':' -f1`
T=`echo "${vals}" | cut -d$':' -f2`
LS=`echo "${S}" | cut -d$'-' -f1`
CS=`echo "${S}" | cut -d$'-' -f2`
LT=`echo "${T}" | cut -d$'-' -f1`
CT=`echo "${T}" | cut -d$'-' -f2`
HQ_NOISE="hqdn3d=luma_spatial=${LS}:chroma_spatial=${CS}:luma_tmp=${LT}:chroma_tmp=${CT}"
hqDesc="3D Denoiser"
FFMPEG_FILTERS=true FFMPEG_FILTERS=true
;; ;;
"SPACE") "SPACE")
@ -507,7 +550,7 @@ evalConf() {
fi fi
LUT="lut3d=${LUT_PATH}" LUT="lut3d=${LUT_PATH}"
isLUT=true lutDesc="3D LUT"
FFMPEG_FILTERS=true FFMPEG_FILTERS=true
;; ;;
"SATPOINT") SATPOINT="-S $(echo "${line}" | cut -d$' ' -f2)" "SATPOINT") SATPOINT="-S $(echo "${line}" | cut -d$' ' -f2)"
@ -518,6 +561,11 @@ evalConf() {
;; ;;
"DESHAKE")
DESHAKE="deshake"
deshakeDesc="Deshake Filter"
FFMPEG_FILTERS=true
;;
"DUAL_ISO") DUAL_ISO=true "DUAL_ISO") DUAL_ISO=true
;; ;;
"BADPIXELS") isBP=true "BADPIXELS") isBP=true
@ -535,7 +583,7 @@ parseArgs() { #Amazing new argument parsing!!!
longArg() { #Creates VAL longArg() { #Creates VAL
ret="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 )) ret="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
} }
while getopts "vh C: o:P: T: i t: m p: s: k r: d: f H: c: n: N: g: w: l: S: u b a: F: R: q K Y M -:" opt; do while getopts "vh C: o:P: T: i t: m p: s: k r: d: f H: c: n: N: Q: g: w: l: S: D u b a: F: R: q K Y M -:" opt; do
#~ echo $opt ${OPTARG} #~ echo $opt ${OPTARG}
case "$opt" in case "$opt" in
-) #Long Arguments -) #Long Arguments
@ -721,7 +769,22 @@ parseArgs() { #Amazing new argument parsing!!!
bVal=$(echo "${vals}" | cut -d"-" -f2) bVal=$(echo "${vals}" | cut -d"-" -f2)
TEMP_NOISE="atadenoise=0a=${aVal}:0b=${bVal}:1a=${aVal}:1b=${bVal}:2a=${aVal}:2b=${bVal}" TEMP_NOISE="atadenoise=0a=${aVal}:0b=${bVal}:1a=${aVal}:1b=${bVal}:2a=${aVal}:2b=${bVal}"
isTemp=true tempDesc="Temporal Denoiser"
FFMPEG_FILTERS=true
;;
Q)
vals=${OPTARG}
S=`echo "${vals}" | cut -d$':' -f1`
T=`echo "${vals}" | cut -d$':' -f2`
LS=`echo "${S}" | cut -d$'-' -f1`
CS=`echo "${S}" | cut -d$'-' -f2`
LT=`echo "${T}" | cut -d$'-' -f1`
CT=`echo "${T}" | cut -d$'-' -f2`
HQ_NOISE="hqdn3d=luma_spatial=${LS}:chroma_spatial=${CS}:luma_tmp=${LT}:chroma_tmp=${CT}"
hqDesc="3D Denoiser"
FFMPEG_FILTERS=true FFMPEG_FILTERS=true
;; ;;
g) g)
@ -770,13 +833,20 @@ parseArgs() { #Amazing new argument parsing!!!
exit 1 exit 1
fi fi
LUT="lut3d=${LUT_PATH}" LUT="lut3d=${LUT_PATH}"
isLUT=true lutDesc="3D LUT"
FFMPEG_FILTERS=true
;; ;;
S) S)
SATPOINT="-S ${OPTARG}" SATPOINT="-S ${OPTARG}"
;; ;;
D)
DESHAKE="deshake"
deshakeDesc="Deshake Filter"
FFMPEG_FILTERS=true
;;
u) u)
DUAL_ISO=true DUAL_ISO=true
;; ;;
@ -1014,13 +1084,18 @@ for ARG in "${FILE_ARGS_ITER[@]}"; do #Go through FILE_ARGS_ITER array, copied f
SCALE=`echo "($(echo "${PROXY_SCALE}" | sed 's/%//') / 100) * 2" | bc -l` #Get scale as factor for halved video, *2 for 50% SCALE=`echo "($(echo "${PROXY_SCALE}" | sed 's/%//') / 100) * 2" | bc -l` #Get scale as factor for halved video, *2 for 50%
setBL=true setBL=true
joinArgs() { local IFS="$1"; shift; echo "$*"; } joinArgs() { local d=$1; shift; echo -n "$1"; shift; printf "%s" "${@/#/$d}"; }
#Construct the FFMPEG filters. #Construct the FFMPEG filters.
if [[ $FFMPEG_FILTERS == true ]]; then
FINAL_SCALE="scale=trunc(iw/2)*${SCALE}:trunc(ih/2)*${SCALE}" FINAL_SCALE="scale=trunc(iw/2)*${SCALE}:trunc(ih/2)*${SCALE}"
V_FILTERS="-vf $(joinArgs , ${TEMP_NOISE} ${LUT})" V_FILTERS="-vf $(joinArgs , ${DESHAKE} ${TEMP_NOISE} ${HQ_NOISE} ${LUT})"
V_FILTERS_PROX="-vf $(joinArgs , ${TEMP_NOISE} ${LUT} ${proxy_scale})" #Proxy filters add the scale component. V_FILTERS_PROX="-vf $(joinArgs , ${DESHAKE} ${TEMP_NOISE} ${HQ_NOISE} ${LUT} ${FINAL_SCALE})" #Proxy filter set adds the scale component.
#Created formatted array of filters, FILTER_ARR.
declare -a compFilters=("${tempDesc}" "${lutDesc}" "${deshakeDesc}" "${hqDesc}")
for v in "${compFilters[@]}"; do if test "$v"; then FILTER_ARR+=("$v"); fi; done
fi
#Evaluate convmlv.conf configuration file for file-specific blocks. #Evaluate convmlv.conf configuration file for file-specific blocks.
evalConf "$LCONFIG" true evalConf "$LCONFIG" true
#Check that things exist. #Check that things exist.
@ -1268,9 +1343,8 @@ for ARG in "${FILE_ARGS_ITER[@]}"; do #Go through FILE_ARGS_ITER array, copied f
done done
} #Progress Bar } #Progress Bar
if [[ $firstFrame == true ]]; then #If 0-0. if [[ $firstFrame == true ]]; then #If 0-0.
rm $(printf "${tmpOut}/${9}_%06d.dng" 1) #Remove frame #1, leaving us only with frame #0. rm $(printf "${tmpOut}/${9}_%06d.dng" 1) 2>/dev/null #Remove frame #1, leaving us only with frame #0.
mv $tmpOut "${7}/0-0" #Move back to 0-0, as if that's how it was developed all along. mv $tmpOut "${7}/0-0" #Move back to 0-0, as if that's how it was developed all along.
fi fi
} }
@ -1480,14 +1554,23 @@ for ARG in "${FILE_ARGS_ITER[@]}"; do #Go through FILE_ARGS_ITER array, copied f
#DEFINE PROCESSING FUNCTIONS #DEFINE PROCESSING FUNCTIONS
#Nasty Hack Part 1: Setting colorspace for dcrawImg must be done differently for non-exr formats to maintain linearity.
if [[ $IMG_FMT == "exr" || $IMG_FMT == "dpx" ]]; then
NASTYHACK="RGB"
else
NASTYHACK="sRGB"
fi
#But even NASTYHACK is one Gamma 2.2 conversion above what it's supposed to be, even if the formats are now consistent.
#So, this is useless...
dcrawOpt() { #Find, develop, and splay raw DNG data as ppm, ready to be processed. dcrawOpt() { #Find, develop, and splay raw DNG data as ppm, ready to be processed.
find "${TMP}" -maxdepth 1 -iname "*.dng" -print0 | sort -z | tr -d "\n" | xargs -0 \ find "${TMP}" -maxdepth 1 -iname "*.dng" -print0 | sort -z | tr -d "\n" | xargs -0 \
dcraw -c -q $DEMO_MODE $FOUR_COLOR -k $BLACK_LEVEL $SATPOINT $BADPIXELS $WHITE -H $HIGHLIGHT_MODE -g $GAMMA $WAVE_NOISE -o $SPACE $DEPTH dcraw -c -q $DEMO_MODE $FOUR_COLOR -k $BLACK_LEVEL $SATPOINT $BADPIXELS $WHITE -H $HIGHLIGHT_MODE -g $GAMMA $WAVE_NOISE -o $SPACE $DEPTH
} #Is prepared to pipe all the files in TMP outwards. } #Is prepared to pipe all the files in TMP outwards.
dcrawImg() { #Find and splay image sequence data as ppm, ready to be processed by ffmpeg. dcrawImg() { #Find and splay image sequence data as ppm, ready to be processed by ffmpeg.
find "${SEQ}" -maxdepth 1 -iname "*.${IMG_FMT}" -print0 | sort -z | xargs -0 -I {} convert '{}' ppm:- find "${SEQ}" -maxdepth 1 -iname "*.${IMG_FMT}" -print0 | sort -z | xargs -0 -I {} convert '{}' -colorspace ${NASTYHACK} ppm:-
} #Finds all images, prints to stdout quickly without any operations using convert. } #Finds all images, prints to stdout, without any operations, using convert. ppm conversion is inevitably slow, however...
mov_main() { mov_main() {
ffmpeg -f image2pipe -vcodec ppm -r $FPS -i pipe:0 \ ffmpeg -f image2pipe -vcodec ppm -r $FPS -i pipe:0 \
@ -1524,17 +1607,30 @@ for ARG in "${FILE_ARGS_ITER[@]}"; do #Go through FILE_ARGS_ITER array, copied f
img_par() { #Takes 20 arguments: {} 2$DEMO_MODE 3$FOUR_COLOR 4$BADPIXELS 5$WHITE 6$HIGHLIGHT_MODE 7$GAMMA 8$WAVE_NOISE 9$DEPTH 10$SEQ 11$TRUNC_ARG 12$IMG_FMT 13$FRAME_END 14$DEPTH_OUT 15$COMPRESS 16$isJPG 17$PROXY_SCALE 18$PROXY 19$BLACK_LEVEL 20$SPACE 21$SATPOINT img_par() { #Takes 20 arguments: {} 2$DEMO_MODE 3$FOUR_COLOR 4$BADPIXELS 5$WHITE 6$HIGHLIGHT_MODE 7$GAMMA 8$WAVE_NOISE 9$DEPTH 10$SEQ 11$TRUNC_ARG 12$IMG_FMT 13$FRAME_END 14$DEPTH_OUT 15$COMPRESS 16$isJPG 17$PROXY_SCALE 18$PROXY 19$BLACK_LEVEL 20$SPACE 21$SATPOINT
count=$(echo $(echo $1 | rev | cut -d "_" -f 1 | rev | cut -d "." -f 1 | grep "[0-9]") | bc) #Instead of count from file, count from name! count=$(echo $(echo $1 | rev | cut -d "_" -f 1 | rev | cut -d "." -f 1 | grep "[0-9]") | bc) #Instead of count from file, count from name!
if [[ ${12} == "dpx" ]]; then NASTYHACK2="-colorspace sRGB"; else NASTYHACK2=""; fi #Nasty Hack, Part 2: IM applies an inverse sRGB curve on DPX, for some reason.
#Trust me, I've tried everything else; but this seems to actually work, so. Bug fixed, I guess...
if [ ${16} == true ]; then if [ ${16} == true ]; then
dcraw -c -q $2 $3 $4 $5 -H $6 -k ${19} ${21} -g $7 $8 -o ${20} $9 $1 | \ dcraw -c -q $2 $3 $4 $5 -H $6 -k ${19} ${21} -g $7 $8 -o ${20} $9 $1 | \
tee >(convert ${14} - ${15} -set colorspace RGB -set colorspace RGB $(printf "${10}/${11}_%06d.${12}" ${count})) | \ tee >(convert ${14} - -set colorspace RGB ${15} $(printf "${10}/${11}_%06d.${12}" ${count})) | \
convert - -quality 90 -set colorspace RGB -resize ${17} -set colorspace RGB $(printf "${18}/${11}_%06d.jpg" ${count}) convert - -set colorspace RGB -quality 80 -set colorspace sRGB -resize ${17} $(printf "${18}/${11}_%06d.jpg" ${count})
#See below, in compression, formats for info on why I'm setting sRGB colorspace for a linear output...
echo -e "\e[2K\rDNG to ${12^^}/JPG: Frame ${count^^}/${13}\c" echo -e "\e[2K\rDNG to ${12^^}/JPG: Frame ${count^^}/${13}\c"
else else
dcraw -c -q $2 $3 $4 $5 -H $6 -k ${19} ${21} -g $7 $8 -o ${20} $9 $1 | \ dcraw -c -q $2 $3 $4 $5 -H $6 -k ${19} ${21} -g $7 $8 -o ${20} $9 $1 | \
convert ${14} - ${15} -set colorspace RGB -set colorspace sRGB $(printf "${10}/${11}_%06d.${12}" ${count}) convert ${14} - -set colorspace RGB ${NASTYHACK2} ${15} $(printf "${10}/${11}_%06d.${12}" ${count})
echo -e "\e[2K\rDNG to ${12^^}: Frame ${count^^}/${13}\c" echo -e "\e[2K\rDNG to ${12^^}: Frame ${count^^}/${13}\c"
fi fi
} #-p /home/sofus/subhome/src/convmlv/care-package/linear_rgb.icc }
#~ For XYZ->sRGB Input (Requires XYZ Identity ICC Profile):
#~ convert ${14} - +profile icm -profile "/home/sofus/subhome/src/convmlv/icc/XYZ-D50-Identity-elle-V4.icc" -set colorspace XYZ \
#~ -color-matrix "3.2404542 -1.5371385 -0.4985314 \
#~ -0.9692660 1.8760108 0.0415560 \
#~ 0.0556434 -0.2040259 1.0572252" \
#~ -set colorspace RGB ${15} $(printf "${10}/${11}_%06d.${12}" ${count})
#~ See http://www.imagemagick.org/discourse-server/viewtopic.php?t=21161
export -f img_par export -f img_par
@ -1583,30 +1679,25 @@ for ARG in "${FILE_ARGS_ITER[@]}"; do #Go through FILE_ARGS_ITER array, copied f
echo -e "\n" echo -e "\n"
#FFMPEG Filter Application: Temporal Denoising, 3D LUTs so far. See construction of $V_FILTERS in PREPARATION. #FFMPEG Filter Application: Temporal Denoising, 3D LUTs, Deshake, hqdn Denoising so far. See construction of $V_FILTERS in PREPARATION.
if [[ $FFMPEG_FILTERS == true ]]; then if [[ $FFMPEG_FILTERS == true ]]; then
tmpFiltered=${TMP}/filtered tmpFiltered=${TMP}/filtered
mkdir $tmpFiltered mkdir $tmpFiltered
#Give correct output. #Give correct output.
if [[ $isTemp == true && $isLUT == true ]]; then echo -e "\e[1mApplying Filters:\e[0m $(joinArgs ", " "${FILTER_ARR[@]}")...\n"
echo -e "\e[1m${TRUNC_ARG}:\e[0m Temporal Denoising and LUT Application...\n"
elif [[ $isTemp == false && $isLUT == true ]]; then
echo -e "\e[1m${TRUNC_ARG}:\e[0m LUT Application...\n"
elif [[ $isTemp == true && $isLUT == false ]]; then
echo -e "\e[1m${TRUNC_ARG}:\e[0m Temporal Denoising...\n"
fi
if [ $IMG_FMT == "exr" ]; then if [ $IMG_FMT == "exr" ]; then
echo -e "Note: EXRs may hang after filtering.\n" echo -e "Note: EXRs may hang after filtering.\n"
img_res=$(identify ${SEQ}/${TRUNC_ARG}_$(printf "%06d" $(echo "$FRAME_START" | bc)).${IMG_FMT} | cut -d$' ' -f3) img_res=$(identify ${SEQ}/${TRUNC_ARG}_$(printf "%06d" $(echo "$FRAME_START" | bc)).${IMG_FMT} | cut -d$' ' -f3)
convert -set colorspace RGB -define stream:buffer-size=0 -set colorspace RGB "${SEQ}/${TRUNC_ARG}_%06d.exr[${FRAME_START}-${FRAME_END}]" ppm:- | \ convert "${SEQ}/${TRUNC_ARG}_%06d.exr[${FRAME_START}-${FRAME_END}]" -define stream:buffer-size=0 -set colorspace RGB ppm:- | \
ffmpeg -f image2pipe -vcodec ppm -s "${img_res}" -r $FPS -loglevel panic -stats -i pipe:0 \ ffmpeg -f image2pipe -vcodec ppm -s "${img_res}" -r $FPS -loglevel panic -stats -i pipe:0 \
$V_FILTERS \ $V_FILTERS \
-vcodec ppm -n -r $FPS -f image2pipe pipe:1 | \ -vcodec ppm -n -r $FPS -f image2pipe pipe:1 | \
convert -depth 16 - -set colorspace RGB -compress piz -set colorspace RGB "${tmpFiltered}/%06d.${IMG_FMT}" #Watch colorspace. convert -depth 16 - -colorspace RGB -compress piz -set colorspace RGB "${tmpFiltered}/%06d.${IMG_FMT}"
#For some reason, this whole process sends EXR's into sRGB. That's why -colorspace RGB is specified. See Nasty Hacks.
else else
ffmpeg -start_number $FRAME_START -f image2 -i "${SEQ}/${TRUNC_ARG}_%06d.${IMG_FMT}" -loglevel panic -stats $V_FILTERS "${tmpFiltered}/%06d.${IMG_FMT}" ffmpeg -start_number $FRAME_START -f image2 -i "${SEQ}/${TRUNC_ARG}_%06d.${IMG_FMT}" -loglevel panic -stats $V_FILTERS "${tmpFiltered}/%06d.${IMG_FMT}"
fi fi
@ -1648,10 +1739,10 @@ for ARG in "${FILE_ARGS_ITER[@]}"; do #Go through FILE_ARGS_ITER array, copied f
V_FILTERS_PROX="-vf $FINAL_SCALE" V_FILTERS_PROX="-vf $FINAL_SCALE"
if [ $isH264 == true ]; then if [ $isH264 == true ]; then
echo -e "\e[1m${TRUNC_ARG}:\e[0m Encoding to ProRes/H.264..." echo -e "\e[1m${TRUNC_ARG}:\e[0m Encoding to ProRes/H.264..."
runSim dcrawImg mov_main mov_prox runSim dcrawOpt mov_main mov_prox #Used to run dcrawImg
else else
echo -e "\e[1m${TRUNC_ARG}:\e[0m Encoding to ProRes..." echo -e "\e[1m${TRUNC_ARG}:\e[0m Encoding to ProRes..."
dcrawImg | mov_main dcrawOpt | mov_main #Used to run dcrawImg
fi fi
echo "" echo ""
fi fi
@ -1660,11 +1751,13 @@ for ARG in "${FILE_ARGS_ITER[@]}"; do #Go through FILE_ARGS_ITER array, copied f
echo -e "\e[1m${TRUNC_ARG}:\e[0m Encoding to H.264..." echo -e "\e[1m${TRUNC_ARG}:\e[0m Encoding to H.264..."
if [ $IMAGES == true ]; then if [ $IMAGES == true ]; then
V_FILTERS_PROX="-vf $FINAL_SCALE" #See above note. V_FILTERS_PROX="-vf $FINAL_SCALE" #See above note.
dcrawImg | mov_prox dcrawOpt | mov_prox #Used to run dcrawImg
else else
dcrawOpt | mov_prox dcrawOpt | mov_prox
fi fi
echo "" echo ""
#Nasty Hack Part 3: We can no longer use dcrawImg; colorspace conversions are a fucking circus.
#Luckily, dcrawOpt isn't that much slower; even faster sometimes. Damn ppm conversion never pulled their weight.
fi fi
#Potentially move DNGs. #Potentially move DNGs.

3
sRange.py
View File

@ -37,6 +37,7 @@ def splitThreadRange(inNum, inThreads) :
if __name__ == "__main__" : if __name__ == "__main__" :
num = int(sys.argv[1]) num = int(sys.argv[1])
threads = int(sys.argv[2]) threads = int(sys.argv[2])
print(*splitThreadRange(num, threads)) if num == 1: print("0-0")
else: print(*splitThreadRange(num, threads))
#mlv_dump cannot use 0-0. #mlv_dump cannot use 0-0.