so-rose
/
convmlv
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 3 Wiki Activity

Fixed the huge colorspace bug (in a weird way - see Nasty Hacks in source). Fixed sRange 1 n error, where no output would be given. Implemented hqdn3d denoiser, and ffmpeg deshake, filters. 

Expanded filter framework; list now formats correctly.
modularize
Sofus Albert Høgsbro Rose 2016-06-27 00:59:14 -04:00
parent 784437b9e1
commit b4e83a6e2f
7 changed files with 230 additions and 178 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
7D_badpixels.txt 100644 → 100755
View File

@ -9,6 +9,10 @@
# DOA - Discovered 3/14/16 # DOA - Discovered 3/14/16
432 481 0 432 481 0
432 600 0
432 601 0
433 601 0
544 109 0
197 1013 0 197 1013 0
167 677 0 167 677 0
343 803 0 343 803 0

5
CHANGELOG
View File

@ -1,4 +1,9 @@
[code] [code]
[b]v1.9.2[/b]: [url=https://bitbucket.org/so-rose/convmlv/src]https://bitbucket.org/so-rose/convmlv/src[/url]
[size=8pt]*Atadenoise Support! It's a great denoising method, used with "-N <A>-<B>" See the documentation for how to configure A and B values.[/size]
[size=8pt]*General FFMPEG Filter Support: The framework for any ffmpeg -vf filters exists now! lut3d and atadenoise are currently implemented.[/size]
[size=8pt]*Bug Fixes: LUTs can now be applied to EXR's! Other bugs fixed too.[/size]
[b]v1.9.1[/b]: [url=https://bitbucket.org/so-rose/convmlv/src]https://bitbucket.org/so-rose/convmlv/src[/url] [b]v1.9.1[/b]: [url=https://bitbucket.org/so-rose/convmlv/src]https://bitbucket.org/so-rose/convmlv/src[/url]
[size=8pt]*Fully Supported Config Files! Check the help page; all options for which this makes sense can be specified in global, local, or file-specific config files.[/size] [size=8pt]*Fully Supported Config Files! Check the help page; all options for which this makes sense can be specified in global, local, or file-specific config files.[/size]
[size=8pt]*File-Specific Blocks: With this, just make one config file w/any special options per file, and you're done. Huge time saver in production situations![/size] [size=8pt]*File-Specific Blocks: With this, just make one config file w/any special options per file, and you're done. Huge time saver in production situations![/size]

144
README.md
View File

@ -15,11 +15,13 @@ Usage:
INFO: INFO:
A script allowing you to develop ML files into workable formats. Many useful options are exposed. A script allowing you to develop ML files into workable formats. Many useful options are exposed.
-->Image Defaults: Compressed 16-bit Linear EXR. --> Image Defaults: Compressed 16-bit Linear EXR.
-->Acceptable Inputs: MLV, RAW, DNG Folder. --> Acceptable Inputs: MLV, RAW, DNG Folder.
-->Option Input: From command line or config file. --> Option Input: From command line or config file.
VERSION: 1.9.2 --> Forum Post: http://www.magiclantern.fm/forum/index.php?topic=16799.
VERSION: 1.9.3
MANUAL DEPENDENCIES: MANUAL DEPENDENCIES:
-mlv_dump: Required. http://www.magiclantern.fm/forum/index.php?topic=7122.0 -mlv_dump: Required. http://www.magiclantern.fm/forum/index.php?topic=7122.0
@ -99,6 +101,14 @@ OPTIONS, RAW DEVELOPMENT:
--> A: 0 to 0.3. B: 0 to 5. A reacts to abrupt noise (splotches), B reacts to noise over time (fast motion causes artifacts). --> A: 0 to 0.3. B: 0 to 5. A reacts to abrupt noise (splotches), B reacts to noise over time (fast motion causes artifacts).
--> Subtle: 0.03-0.04. High: 0.15-0.04. High, Predictable Motion: 0.15-0.07 --> Subtle: 0.03-0.04. High: 0.15-0.04. High, Predictable Motion: 0.15-0.07
-Q [i-i:i-i] HQ_NOISE - Apply 3D denoising filter.
--> In depth explanation: https://mattgadient.com/2013/06/29/in-depth-look-at-de-noising-in-handbrake-with-imagevideo-examples/ .
--> Spacial/Temporal (S/T). S will soften/blur/smooth, T will remove noise without doing that but may create artifacts.
--> Luma/Chroma (L/C). L is the detail, C is the color. Each one's denoising may be manipulated Spacially or Temporally.
--> Options: <LS>-<CS>:<LT>-<CT>
--> Weak: 2-1:2-3. Medium: 3-2:2-3. Strong: 7-7:5-5
-g [0:4] SPACE - Output color transformation. -g [0:4] SPACE - Output color transformation.
--> 0: Linear. 1: 2.2 (Adobe RGB). 2: 1.8 (ProPhoto RGB). 3: sRGB. 4: BT.709. --> 0: Linear. 1: 2.2 (Adobe RGB). 2: 1.8 (ProPhoto RGB). 3: sRGB. 4: BT.709.
@ -112,7 +122,7 @@ OPTIONS, COLOR:
-l <path> LUT - Specify a LUT to apply. -l <path> LUT - Specify a LUT to apply.
--> Supports cube, 3dl, dat, m3d. --> Supports cube, 3dl, dat, m3d.
-S [int] SATPOINT - Specify the 14-bit saturation point of your camera. -S [int] SATPOINT - Specify the 14-bit saturation point of your camera. You don't usually want to.
--> Lower if -H1 yields purple highlights. Must be correct for highlight reconstruction. --> Lower if -H1 yields purple highlights. Must be correct for highlight reconstruction.
--> Determine using the max value of 'dcraw -D -j -4 -T' --> Determine using the max value of 'dcraw -D -j -4 -T'
@ -122,6 +132,9 @@ OPTIONS, COLOR:
OPTIONS, FEATURES: OPTIONS, FEATURES:
-D DESHAKE - Stabilize the video using the wonderful ffmpeg "deshake" module.
--> You'll probably wish to crop/scale the output in editing, to avoid edge artifacts.
-u DUAL_ISO - Process as dual ISO. -u DUAL_ISO - Process as dual ISO.
-b BADPIXELS - Fix focus pixels issue using dfort's script. -b BADPIXELS - Fix focus pixels issue using dfort's script.
@ -134,7 +147,8 @@ OPTIONS, FEATURES:
--> If the file extension is '.darkframe', the file will be used as the preaveraged dark frame. --> If the file extension is '.darkframe', the file will be used as the preaveraged dark frame.
-R <path> dark_out - Specify to create a .darkframe file from passed in MLV. -R <path> dark_out - Specify to create a .darkframe file from passed in MLV.
--> Outputs <arg>.darkframe file to <path>. --> Usage: 'convmlv -R <path> <input>.MLV'
--> Averages <input>.MLV to create <path>.darkframe.
--> THE .darkframe EXTENSION IS ADDED FOR YOU. --> THE .darkframe EXTENSION IS ADDED FOR YOU.
@ -151,113 +165,45 @@ OPTIONS, INFO:
--> There's no automatic way to install these. See http://www.magiclantern.fm/forum/index.php?topic=16799.0 . --> There's no automatic way to install these. See http://www.magiclantern.fm/forum/index.php?topic=16799.0 .
CONFIG FILE: CONFIG FILE:
You do not need to type in all the arguments each time: Config files, another way to specify options, can save you time & lend Config files, another way to specify options, can save you time & lend you convenience in production situations.
you convenience in production situations.
GLOBAL: /home/sofus/convmlv.conf GLOBAL: /home/sofus/convmlv.conf
LOCAL: Specify -C/--config. LOCAL: Specify -C/--config.
Some options have an uppercased VARNAME, ex. OUTDIR. In a convmlv config file, you can specify this option
in the following format, line by line:
<VARNAME> <VALUE>
If the value is a true/false flag, simply specifying VARNAME is enough. Otherwise, normal rules for the value applies. SYNTAX:
Most options listed above have an uppercased VARNAME, ex. OUTDIR. Yu can specify such options in config files, as such:
Options override each other as such: <VARNAME> <VALUE>
-LOCAL overrides GLOBAL config.
-Passed arguments override both configs. One option per line only. Indentation by tabs or spaces is allowed, but not enforced.
-Lines starting with # are comments.
-Name a config using the VARNAME: CONFIG_NAME <name> Comments Lines starting with # are comments.
You may name a config using:
CONFIG_NAME <name>
Flags If the value is a true/false flag (ex. IMAGE), simply specifying VARNAME is enough. THere is no VALUE.
OPTION ORDER OF PRECEDENCE Options override each other as such:
-LOCAL options overwrite GLOBAL options.
-COMMAND LINE options overwrite LOCAL & GLOBAL options.
-FILE SPECIFIC options overwrite ALL ABOVE options.
File-Specific Block: A LOCAL config file lets you specify options for specific input names: File-Specific Block: A LOCAL config file lets you specify options for specific input names:
/ <TRUNCATED INPUTNAME> / <TRUNCATED INPUTNAME>
...specify options ...options here will only be
* *
File-Specific Blocks override all other options. This allows one to create You must use the truncated (no .mlv or .raw) input name after the /. Nested blocks will fail.
a single config file to batch-develop several input files/folders at once, after deciding how each one should
look/be configured individually.
Notes on Usage: With a single config file, you can control the development options of multiple inputs as specifically and/or generically
-You must use the truncated (no .mlv or .raw) input name after the /. as you want. Batch developing everything can then be done with a single, powerful commmand.
-No nested blocks.
-Indentation by tabs or spaces is allowed, but not enforced. -w [0:2] WHITE - This is a modal white balance setting.
--> 0: Auto WB. 1: Camera WB (default). 2: No Change.
-l <path> LUT - Specify a LUT to apply.
--> Supports cube, 3dl, dat, m3d.
--> LUT cannot be applied to EXR sequences.
-S [int] SATPOINT - Specify the 14-bit saturation point of your camera.
--> Lower if -H1 yields purple highlights. Must be correct for highlight reconstruction.
--> Determine using the max value of 'dcraw -D -j -4 -T'
--white-speed [int] WHITE_SPD - Samples used to calculate AWB
--allow-white-clip WHITE_CLIP - Let White Balance multipliers clip.
OPTIONS, FEATURES:
-u DUAL_ISO - Process as dual ISO.
-b BADPIXELS - Fix focus pixels issue using dfort's script. Contact me with any feedback or questions at convmlv@sofusrose.com, or PM me (so-rose) on the ML forums!
-a <path> BADPIXEL_PATH - Use your own .badpixels file.
--> How to: http://www.dl-c.com/board/viewtopic.php?f=4&t=686
-F <path> DARKFRAME - This is the path to a "dark frame MLV"; effective for noise reduction.
--> How to: Record 5 sec w/lens cap on & same settings as footage. Pass MLV in here.
--> If the file extension is '.darkframe', the file will be used as the preaveraged dark frame.
-R <path> dark_out - Specify to create a .darkframe file from passed in MLV.
--> Outputs <arg>.darkframe file to <path>.
--> THE .darkframe EXTENSION IS ADDED FOR YOU.
OPTIONS, INFO:
-q Output MLV settings.
-K Debian Package Deps - Output package dependecies.
--> Install (Debian only): sudo apt-get install $ (./convmlv -K)
-Y Python Deps - Lists Python dependencies. Works directly with pip.
-->Install (Linux): sudo pip3 install $ (./convmlv -Y)
-N Manual Deps - Lists manual dependencies, which must be downloaded by hand.
--> There's no automatic way to install these. See http://www.magiclantern.fm/forum/index.php?topic=16799.0 .
CONFIG FILE:
You do not need to type in all the arguments each time: Config files, another way to specify options, can save you time & lend
you convenience in production situations.
GLOBAL: /home/sofus/convmlv.conf
LOCAL: Specify -C/--config.
Some options have an uppercased VARNAME, ex. OUTDIR. In a convmlv config file, you can specify this option
in the following format, line by line:
<VARNAME> <VALUE>
If the value is a true/false flag, simply specifying VARNAME is enough. Otherwise, normal rules for the value applies.
Options override each other as such:
-LOCAL overrides GLOBAL config.
-Passed arguments override both configs.
-Lines starting with # are comments.
-Name a config using the VARNAME: CONFIG_NAME <name>
File-Specific Block: A LOCAL config file lets you specify options for specific input names:
/ <TRUNCATED INPUTNAME>
...specify options
*
File-Specific Blocks override all other options. This allows one to create
a single config file to batch-develop several input files/folders at once, after deciding how each one should
look/be configured individually.
Notes on Usage:
-You must use the truncated (no .mlv or .raw) input name after the /.
-No nested blocks.
-Indentation by tabs or spaces is allowed, but not enforced.
``` ```

9
configs/atadenoise-test.conf
View File

@ -20,22 +20,22 @@ PYTHON python3
IMAGE IMAGE
IMG_FMT 0 IMG_FMT 0
#~ MOVIE MOVIE
PROXY 0 PROXY 0
#~ PROXY_SCALE 50% #~ PROXY_SCALE 50%
KEEP_DNGS KEEP_DNGS
FRAME_RANGE 300-350 FRAME_RANGE 300-310
#UNCOMP UNCOMP
### <------------------ RAW OPTIONS ------------------> ### <------------------ RAW OPTIONS ------------------>
DEMO_MODE 2 DEMO_MODE 2
HIGHLIGHT_MODE 0 HIGHLIGHT_MODE 0
#~ TEMP_NOISE 0.15-0.04 #~ TEMP_NOISE 0.15-0.04
#~ HQ_NOISE 7-7:5-5
SPACE 0 SPACE 0
@ -44,6 +44,7 @@ SPACE 0
WHITE 1 WHITE 1
### <------------------ FEATURE OPTIONS ------------------> ### <------------------ FEATURE OPTIONS ------------------>
DESHAKE
#DARKFRAME ./df.darkframe #DARKFRAME ./df.darkframe
#~ LUT /home/sofus/subhome/src/convmlv/test_footage/girl_lut_test.cube #~ LUT /home/sofus/subhome/src/convmlv/test_footage/girl_lut_test.cube

2
configs/template.txt
View File

@ -40,6 +40,7 @@ HIGHLIGHT_MODE 0
CHROMA_SMOOTH 1 CHROMA_SMOOTH 1
WAVE_NOISE 50 WAVE_NOISE 50
TEMP_NOISE 0.03-0.04 TEMP_NOISE 0.03-0.04
HQ_NOISE 2-1:2-3
SPACE 3 SPACE 3
#SHALLOW #SHALLOW
@ -56,6 +57,7 @@ WHITE 0
#WHITE_CLIP #WHITE_CLIP
### <------------------ FEATURE OPTIONS ------------------> ### <------------------ FEATURE OPTIONS ------------------>
#DESHAKE
#DUAL_ISO #DUAL_ISO
BADPIXELS BADPIXELS

223
convmlv.sh
View File

@ -3,13 +3,14 @@
#TODO: #TODO:
#~ Stats for .RAW files. #~ Stats for .RAW files.
#~ Integrate anti-vertical banding. May require being able to use multiple darkframe files. #~ Integrate anti-vertical banding. May require being able to use multiple darkframe files.
#~ Atadenoise integration.
#~ Better Preview: #~ Better Preview:
#~ --> A different module (like -e) for live viewing of footage, under convmlv settings. Danne is working on this :). #~ --> A different module (like -e) for live viewing of footage, under convmlv settings. Danne is working on this :).
#BUG: Relative OUTDIR makes baxpixel generation fail if ./mlv2badpixels.sh doesn't exist. Fixed on Linux only. #BUG: Relative OUTDIR makes baxpixel generation fail if ./mlv2badpixels.sh doesn't exist. Fixed on Linux only.
#BUG: Colorspace Bug Affecting all but EXR output. #CONCERN: Weirdness with color spaces in general. No impact to user; just some weirdly placed sRGB conversions if'ed by DPX.
#~ --> See Nasty Hacks Part 1 - 3. FML :O.
#~ I'll feel better about this after quite a while
#~ The MIT License (MIT) #~ The MIT License (MIT)
@ -35,7 +36,7 @@
#~ SOFTWARE. #~ SOFTWARE.
#BASIC VARS #BASIC VARS
VERSION="1.9.2" #Version string. VERSION="1.9.3" #Version string.
INPUT_ARGS=$(echo "$@") #The original input argument string. INPUT_ARGS=$(echo "$@") #The original input argument string.
if [[ $OSTYPE == "linux-gnu" ]]; then if [[ $OSTYPE == "linux-gnu" ]]; then
@ -130,9 +131,13 @@ setDefaults() { #Set all the default variables. Run here, and also after each AR
#FFMPEG Filters #FFMPEG Filters
FFMPEG_FILTERS=false #Whether or not FFMPEG filters are going to be used. FFMPEG_FILTERS=false #Whether or not FFMPEG filters are going to be used.
TEMP_NOISE="" #Temporal noise reduction. TEMP_NOISE="" #Temporal noise reduction.
isTemp=false tempDesc=""
LUT="" LUT=""
isLUT=false lutDesc=""
DESHAKE=""
deshakeDesc=""
HQ_NOISE="" #hqdn3d noise reduction.
hqDesc=""
} }
setDefaults #Run now, but also later. setDefaults #Run now, but also later.
@ -144,9 +149,11 @@ Usage:
INFO: INFO:
A script allowing you to develop ML files into workable formats. Many useful options are exposed. A script allowing you to develop ML files into workable formats. Many useful options are exposed.
-->Image Defaults: Compressed 16-bit Linear EXR. --> Image Defaults: Compressed 16-bit Linear EXR.
-->Acceptable Inputs: MLV, RAW, DNG Folder. --> Acceptable Inputs: MLV, RAW, DNG Folder.
-->Option Input: From command line or config file. --> Option Input: From command line or config file.
--> Forum Post: http://www.magiclantern.fm/forum/index.php?topic=16799.
$(echo -e "VERSION: ${VERSION}") $(echo -e "VERSION: ${VERSION}")
@ -228,6 +235,14 @@ OPTIONS, RAW DEVELOPMENT:
--> A: 0 to 0.3. B: 0 to 5. A reacts to abrupt noise (splotches), B reacts to noise over time (fast motion causes artifacts). --> A: 0 to 0.3. B: 0 to 5. A reacts to abrupt noise (splotches), B reacts to noise over time (fast motion causes artifacts).
--> Subtle: 0.03-0.04. High: 0.15-0.04. High, Predictable Motion: 0.15-0.07 --> Subtle: 0.03-0.04. High: 0.15-0.04. High, Predictable Motion: 0.15-0.07
-Q [i-i:i-i] HQ_NOISE - Apply 3D denoising filter.
--> In depth explanation: https://mattgadient.com/2013/06/29/in-depth-look-at-de-noising-in-handbrake-with-imagevideo-examples/ .
--> Spacial/Temporal (S/T). S will soften/blur/smooth, T will remove noise without doing that but may create artifacts.
--> Luma/Chroma (L/C). L is the detail, C is the color. Each one's denoising may be manipulated Spacially or Temporally.
--> Options: <LS>-<CS>:<LT>-<CT>
--> Weak: 2-1:2-3. Medium: 3-2:2-3. Strong: 7-7:5-5
-g [0:4] SPACE - Output color transformation. -g [0:4] SPACE - Output color transformation.
--> 0: Linear. 1: 2.2 (Adobe RGB). 2: 1.8 (ProPhoto RGB). 3: sRGB. 4: BT.709. --> 0: Linear. 1: 2.2 (Adobe RGB). 2: 1.8 (ProPhoto RGB). 3: sRGB. 4: BT.709.
@ -241,7 +256,7 @@ OPTIONS, COLOR:
-l <path> LUT - Specify a LUT to apply. -l <path> LUT - Specify a LUT to apply.
--> Supports cube, 3dl, dat, m3d. --> Supports cube, 3dl, dat, m3d.
-S [int] SATPOINT - Specify the 14-bit saturation point of your camera. -S [int] SATPOINT - Specify the 14-bit saturation point of your camera. You don't usually want to.
--> Lower if -H1 yields purple highlights. Must be correct for highlight reconstruction. --> Lower if -H1 yields purple highlights. Must be correct for highlight reconstruction.
--> Determine using the max value of 'dcraw -D -j -4 -T' --> Determine using the max value of 'dcraw -D -j -4 -T'
@ -251,6 +266,9 @@ OPTIONS, COLOR:
OPTIONS, FEATURES: OPTIONS, FEATURES:
-D DESHAKE - Stabilize the video using the wonderful ffmpeg "deshake" module.
--> You'll probably wish to crop/scale the output in editing, to avoid edge artifacts.
-u DUAL_ISO - Process as dual ISO. -u DUAL_ISO - Process as dual ISO.
-b BADPIXELS - Fix focus pixels issue using dfort's script. -b BADPIXELS - Fix focus pixels issue using dfort's script.
@ -263,7 +281,8 @@ OPTIONS, FEATURES:
--> If the file extension is '.darkframe', the file will be used as the preaveraged dark frame. --> If the file extension is '.darkframe', the file will be used as the preaveraged dark frame.
-R <path> dark_out - Specify to create a .darkframe file from passed in MLV. -R <path> dark_out - Specify to create a .darkframe file from passed in MLV.
--> Outputs <arg>.darkframe file to <path>. --> Usage: 'convmlv -R <path> <input>.MLV'
--> Averages <input>.MLV to create <path>.darkframe.
--> THE .darkframe EXTENSION IS ADDED FOR YOU. --> THE .darkframe EXTENSION IS ADDED FOR YOU.
@ -280,38 +299,47 @@ OPTIONS, INFO:
--> There's no automatic way to install these. See http://www.magiclantern.fm/forum/index.php?topic=16799.0 . --> There's no automatic way to install these. See http://www.magiclantern.fm/forum/index.php?topic=16799.0 .
CONFIG FILE: CONFIG FILE:
You do not need to type in all the arguments each time: Config files, another way to specify options, can save you time & lend Config files, another way to specify options, can save you time & lend you convenience in production situations.
you convenience in production situations.
GLOBAL: $HOME/convmlv.conf $(echo -e "\e[1mGLOBAL\e[0m"): $HOME/convmlv.conf
LOCAL: Specify -C/--config. $(echo -e "\e[1mLOCAL\e[0m"): Specify -C/--config.
Some options have an uppercased VARNAME, ex. OUTDIR. In a convmlv config file, you can specify this option
in the following format, line by line:
<VARNAME> <VALUE>
If the value is a true/false flag, simply specifying VARNAME is enough. Otherwise, normal rules for the value applies.
Options override each other as such:
-LOCAL overrides GLOBAL config.
-Passed arguments override both configs.
-Lines starting with # are comments.
-Name a config using the VARNAME: CONFIG_NAME <name>
File-Specific Block: A LOCAL config file lets you specify options for specific input names: $(echo -e "\e[1mSYNTAX:\e[0m")
Most options listed above have an uppercased VARNAME, ex. OUTDIR. Yu can specify such options in config files, as such:
<VARNAME> <VALUE>
One option per line only. Indentation by tabs or spaces is allowed, but not enforced.
$(echo -e "\e[1mComments\e[0m") Lines starting with # are comments.
You may name a config using:
CONFIG_NAME <name>
$(echo -e "\e[1mFlags\e[0m") If the value is a true/false flag (ex. IMAGE), simply specifying VARNAME is enough. THere is no VALUE.
$(echo -e "\e[1mOPTION ORDER OF PRECEDENCE\e[0m") Options override each other as such:
-LOCAL options overwrite GLOBAL options.
-COMMAND LINE options overwrite LOCAL & GLOBAL options.
-FILE SPECIFIC options overwrite ALL ABOVE options.
$(echo -e "\e[1mFile-Specific Block\e[0m"): A LOCAL config file lets you specify options for specific input names:
/ <TRUNCATED INPUTNAME> / <TRUNCATED INPUTNAME>
...specify options ...options here will only be
* *
$(echo -e "\e[1mFile-Specific Blocks override all other options.\e[0m") This allows one to create You must use the truncated (no .mlv or .raw) input name after the /. Nested blocks will fail.
a single config file to batch-develop several input files/folders at once, after deciding how each one should
look/be configured individually.
Notes on Usage: With a single config file, you can control the development options of multiple inputs as specifically and/or generically
-You must use the truncated (no .mlv or .raw) input name after the /. as you want. Batch developing everything can then be done with a single, powerful commmand.
-No nested blocks.
-Indentation by tabs or spaces is allowed, but not enforced.
Contact me with any feedback or questions at convmlv@sofusrose.com, or PM me (so-rose) on the ML forums!
EOF EOF
} }
@ -456,7 +484,22 @@ evalConf() {
bVal=$(echo "${vals}" | cut -d"-" -f2) bVal=$(echo "${vals}" | cut -d"-" -f2)
TEMP_NOISE="atadenoise=0a=${aVal}:0b=${bVal}:1a=${aVal}:1b=${bVal}:2a=${aVal}:2b=${bVal}" TEMP_NOISE="atadenoise=0a=${aVal}:0b=${bVal}:1a=${aVal}:1b=${bVal}:2a=${aVal}:2b=${bVal}"
isTemp=true tempDesc="Temporal Denoiser"
FFMPEG_FILTERS=true
;;
"HQ_NOISE")
vals="$(echo "${line}" | cut -d$' ' -f2)"
S=`echo "${vals}" | cut -d$':' -f1`
T=`echo "${vals}" | cut -d$':' -f2`
LS=`echo "${S}" | cut -d$'-' -f1`
CS=`echo "${S}" | cut -d$'-' -f2`
LT=`echo "${T}" | cut -d$'-' -f1`
CT=`echo "${T}" | cut -d$'-' -f2`
HQ_NOISE="hqdn3d=luma_spatial=${LS}:chroma_spatial=${CS}:luma_tmp=${LT}:chroma_tmp=${CT}"
hqDesc="3D Denoiser"
FFMPEG_FILTERS=true FFMPEG_FILTERS=true
;; ;;
"SPACE") "SPACE")
@ -507,7 +550,7 @@ evalConf() {
fi fi
LUT="lut3d=${LUT_PATH}" LUT="lut3d=${LUT_PATH}"
isLUT=true lutDesc="3D LUT"
FFMPEG_FILTERS=true FFMPEG_FILTERS=true
;; ;;
"SATPOINT") SATPOINT="-S $(echo "${line}" | cut -d$' ' -f2)" "SATPOINT") SATPOINT="-S $(echo "${line}" | cut -d$' ' -f2)"
@ -518,6 +561,11 @@ evalConf() {
;; ;;
"DESHAKE")
DESHAKE="deshake"
deshakeDesc="Deshake Filter"
FFMPEG_FILTERS=true
;;
"DUAL_ISO") DUAL_ISO=true "DUAL_ISO") DUAL_ISO=true
;; ;;
"BADPIXELS") isBP=true "BADPIXELS") isBP=true
@ -535,7 +583,7 @@ parseArgs() { #Amazing new argument parsing!!!
longArg() { #Creates VAL longArg() { #Creates VAL
ret="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 )) ret="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
} }
while getopts "vh C: o:P: T: i t: m p: s: k r: d: f H: c: n: N: g: w: l: S: u b a: F: R: q K Y M -:" opt; do while getopts "vh C: o:P: T: i t: m p: s: k r: d: f H: c: n: N: Q: g: w: l: S: D u b a: F: R: q K Y M -:" opt; do
#~ echo $opt ${OPTARG} #~ echo $opt ${OPTARG}
case "$opt" in case "$opt" in
-) #Long Arguments -) #Long Arguments
@ -721,7 +769,22 @@ parseArgs() { #Amazing new argument parsing!!!
bVal=$(echo "${vals}" | cut -d"-" -f2) bVal=$(echo "${vals}" | cut -d"-" -f2)
TEMP_NOISE="atadenoise=0a=${aVal}:0b=${bVal}:1a=${aVal}:1b=${bVal}:2a=${aVal}:2b=${bVal}" TEMP_NOISE="atadenoise=0a=${aVal}:0b=${bVal}:1a=${aVal}:1b=${bVal}:2a=${aVal}:2b=${bVal}"
isTemp=true tempDesc="Temporal Denoiser"
FFMPEG_FILTERS=true
;;
Q)
vals=${OPTARG}
S=`echo "${vals}" | cut -d$':' -f1`
T=`echo "${vals}" | cut -d$':' -f2`
LS=`echo "${S}" | cut -d$'-' -f1`
CS=`echo "${S}" | cut -d$'-' -f2`
LT=`echo "${T}" | cut -d$'-' -f1`
CT=`echo "${T}" | cut -d$'-' -f2`
HQ_NOISE="hqdn3d=luma_spatial=${LS}:chroma_spatial=${CS}:luma_tmp=${LT}:chroma_tmp=${CT}"
hqDesc="3D Denoiser"
FFMPEG_FILTERS=true FFMPEG_FILTERS=true
;; ;;
g) g)
@ -770,13 +833,20 @@ parseArgs() { #Amazing new argument parsing!!!
exit 1 exit 1
fi fi
LUT="lut3d=${LUT_PATH}" LUT="lut3d=${LUT_PATH}"
isLUT=true lutDesc="3D LUT"
FFMPEG_FILTERS=true
;; ;;
S) S)
SATPOINT="-S ${OPTARG}" SATPOINT="-S ${OPTARG}"
;; ;;
D)
DESHAKE="deshake"
deshakeDesc="Deshake Filter"
FFMPEG_FILTERS=true
;;
u) u)
DUAL_ISO=true DUAL_ISO=true
;; ;;
@ -1014,13 +1084,18 @@ for ARG in "${FILE_ARGS_ITER[@]}"; do #Go through FILE_ARGS_ITER array, copied f
SCALE=`echo "($(echo "${PROXY_SCALE}" | sed 's/%//') / 100) * 2" | bc -l` #Get scale as factor for halved video, *2 for 50% SCALE=`echo "($(echo "${PROXY_SCALE}" | sed 's/%//') / 100) * 2" | bc -l` #Get scale as factor for halved video, *2 for 50%
setBL=true setBL=true
joinArgs() { local IFS="$1"; shift; echo "$*"; } joinArgs() { local d=$1; shift; echo -n "$1"; shift; printf "%s" "${@/#/$d}"; }
#Construct the FFMPEG filters. #Construct the FFMPEG filters.
FINAL_SCALE="scale=trunc(iw/2)*${SCALE}:trunc(ih/2)*${SCALE}" if [[ $FFMPEG_FILTERS == true ]]; then
V_FILTERS="-vf $(joinArgs , ${TEMP_NOISE} ${LUT})" FINAL_SCALE="scale=trunc(iw/2)*${SCALE}:trunc(ih/2)*${SCALE}"
V_FILTERS_PROX="-vf $(joinArgs , ${TEMP_NOISE} ${LUT} ${proxy_scale})" #Proxy filters add the scale component. V_FILTERS="-vf $(joinArgs , ${DESHAKE} ${TEMP_NOISE} ${HQ_NOISE} ${LUT})"
V_FILTERS_PROX="-vf $(joinArgs , ${DESHAKE} ${TEMP_NOISE} ${HQ_NOISE} ${LUT} ${FINAL_SCALE})" #Proxy filter set adds the scale component.
#Created formatted array of filters, FILTER_ARR.
declare -a compFilters=("${tempDesc}" "${lutDesc}" "${deshakeDesc}" "${hqDesc}")
for v in "${compFilters[@]}"; do if test "$v"; then FILTER_ARR+=("$v"); fi; done
fi
#Evaluate convmlv.conf configuration file for file-specific blocks. #Evaluate convmlv.conf configuration file for file-specific blocks.
evalConf "$LCONFIG" true evalConf "$LCONFIG" true
#Check that things exist. #Check that things exist.
@ -1268,9 +1343,8 @@ for ARG in "${FILE_ARGS_ITER[@]}"; do #Go through FILE_ARGS_ITER array, copied f
done done
} #Progress Bar } #Progress Bar
if [[ $firstFrame == true ]]; then #If 0-0. if [[ $firstFrame == true ]]; then #If 0-0.
rm $(printf "${tmpOut}/${9}_%06d.dng" 1) #Remove frame #1, leaving us only with frame #0. rm $(printf "${tmpOut}/${9}_%06d.dng" 1) 2>/dev/null #Remove frame #1, leaving us only with frame #0.
mv $tmpOut "${7}/0-0" #Move back to 0-0, as if that's how it was developed all along. mv $tmpOut "${7}/0-0" #Move back to 0-0, as if that's how it was developed all along.
fi fi
} }
@ -1480,14 +1554,23 @@ for ARG in "${FILE_ARGS_ITER[@]}"; do #Go through FILE_ARGS_ITER array, copied f
#DEFINE PROCESSING FUNCTIONS #DEFINE PROCESSING FUNCTIONS
#Nasty Hack Part 1: Setting colorspace for dcrawImg must be done differently for non-exr formats to maintain linearity.
if [[ $IMG_FMT == "exr" || $IMG_FMT == "dpx" ]]; then
NASTYHACK="RGB"
else
NASTYHACK="sRGB"
fi
#But even NASTYHACK is one Gamma 2.2 conversion above what it's supposed to be, even if the formats are now consistent.
#So, this is useless...
dcrawOpt() { #Find, develop, and splay raw DNG data as ppm, ready to be processed. dcrawOpt() { #Find, develop, and splay raw DNG data as ppm, ready to be processed.
find "${TMP}" -maxdepth 1 -iname "*.dng" -print0 | sort -z | tr -d "\n" | xargs -0 \ find "${TMP}" -maxdepth 1 -iname "*.dng" -print0 | sort -z | tr -d "\n" | xargs -0 \
dcraw -c -q $DEMO_MODE $FOUR_COLOR -k $BLACK_LEVEL $SATPOINT $BADPIXELS $WHITE -H $HIGHLIGHT_MODE -g $GAMMA $WAVE_NOISE -o $SPACE $DEPTH dcraw -c -q $DEMO_MODE $FOUR_COLOR -k $BLACK_LEVEL $SATPOINT $BADPIXELS $WHITE -H $HIGHLIGHT_MODE -g $GAMMA $WAVE_NOISE -o $SPACE $DEPTH
} #Is prepared to pipe all the files in TMP outwards. } #Is prepared to pipe all the files in TMP outwards.
dcrawImg() { #Find and splay image sequence data as ppm, ready to be processed by ffmpeg. dcrawImg() { #Find and splay image sequence data as ppm, ready to be processed by ffmpeg.
find "${SEQ}" -maxdepth 1 -iname "*.${IMG_FMT}" -print0 | sort -z | xargs -0 -I {} convert '{}' ppm:- find "${SEQ}" -maxdepth 1 -iname "*.${IMG_FMT}" -print0 | sort -z | xargs -0 -I {} convert '{}' -colorspace ${NASTYHACK} ppm:-
} #Finds all images, prints to stdout quickly without any operations using convert. } #Finds all images, prints to stdout, without any operations, using convert. ppm conversion is inevitably slow, however...
mov_main() { mov_main() {
ffmpeg -f image2pipe -vcodec ppm -r $FPS -i pipe:0 \ ffmpeg -f image2pipe -vcodec ppm -r $FPS -i pipe:0 \
@ -1524,17 +1607,30 @@ for ARG in "${FILE_ARGS_ITER[@]}"; do #Go through FILE_ARGS_ITER array, copied f
img_par() { #Takes 20 arguments: {} 2$DEMO_MODE 3$FOUR_COLOR 4$BADPIXELS 5$WHITE 6$HIGHLIGHT_MODE 7$GAMMA 8$WAVE_NOISE 9$DEPTH 10$SEQ 11$TRUNC_ARG 12$IMG_FMT 13$FRAME_END 14$DEPTH_OUT 15$COMPRESS 16$isJPG 17$PROXY_SCALE 18$PROXY 19$BLACK_LEVEL 20$SPACE 21$SATPOINT img_par() { #Takes 20 arguments: {} 2$DEMO_MODE 3$FOUR_COLOR 4$BADPIXELS 5$WHITE 6$HIGHLIGHT_MODE 7$GAMMA 8$WAVE_NOISE 9$DEPTH 10$SEQ 11$TRUNC_ARG 12$IMG_FMT 13$FRAME_END 14$DEPTH_OUT 15$COMPRESS 16$isJPG 17$PROXY_SCALE 18$PROXY 19$BLACK_LEVEL 20$SPACE 21$SATPOINT
count=$(echo $(echo $1 | rev | cut -d "_" -f 1 | rev | cut -d "." -f 1 | grep "[0-9]") | bc) #Instead of count from file, count from name! count=$(echo $(echo $1 | rev | cut -d "_" -f 1 | rev | cut -d "." -f 1 | grep "[0-9]") | bc) #Instead of count from file, count from name!
if [[ ${12} == "dpx" ]]; then NASTYHACK2="-colorspace sRGB"; else NASTYHACK2=""; fi #Nasty Hack, Part 2: IM applies an inverse sRGB curve on DPX, for some reason.
#Trust me, I've tried everything else; but this seems to actually work, so. Bug fixed, I guess...
if [ ${16} == true ]; then if [ ${16} == true ]; then
dcraw -c -q $2 $3 $4 $5 -H $6 -k ${19} ${21} -g $7 $8 -o ${20} $9 $1 | \ dcraw -c -q $2 $3 $4 $5 -H $6 -k ${19} ${21} -g $7 $8 -o ${20} $9 $1 | \
tee >(convert ${14} - ${15} -set colorspace RGB -set colorspace RGB $(printf "${10}/${11}_%06d.${12}" ${count})) | \ tee >(convert ${14} - -set colorspace RGB ${15} $(printf "${10}/${11}_%06d.${12}" ${count})) | \
convert - -quality 90 -set colorspace RGB -resize ${17} -set colorspace RGB $(printf "${18}/${11}_%06d.jpg" ${count}) convert - -set colorspace RGB -quality 80 -set colorspace sRGB -resize ${17} $(printf "${18}/${11}_%06d.jpg" ${count})
#See below, in compression, formats for info on why I'm setting sRGB colorspace for a linear output...
echo -e "\e[2K\rDNG to ${12^^}/JPG: Frame ${count^^}/${13}\c" echo -e "\e[2K\rDNG to ${12^^}/JPG: Frame ${count^^}/${13}\c"
else else
dcraw -c -q $2 $3 $4 $5 -H $6 -k ${19} ${21} -g $7 $8 -o ${20} $9 $1 | \ dcraw -c -q $2 $3 $4 $5 -H $6 -k ${19} ${21} -g $7 $8 -o ${20} $9 $1 | \
convert ${14} - ${15} -set colorspace RGB -set colorspace sRGB $(printf "${10}/${11}_%06d.${12}" ${count}) convert ${14} - -set colorspace RGB ${NASTYHACK2} ${15} $(printf "${10}/${11}_%06d.${12}" ${count})
echo -e "\e[2K\rDNG to ${12^^}: Frame ${count^^}/${13}\c" echo -e "\e[2K\rDNG to ${12^^}: Frame ${count^^}/${13}\c"
fi fi
} #-p /home/sofus/subhome/src/convmlv/care-package/linear_rgb.icc }
#~ For XYZ->sRGB Input (Requires XYZ Identity ICC Profile):
#~ convert ${14} - +profile icm -profile "/home/sofus/subhome/src/convmlv/icc/XYZ-D50-Identity-elle-V4.icc" -set colorspace XYZ \
#~ -color-matrix "3.2404542 -1.5371385 -0.4985314 \
#~ -0.9692660 1.8760108 0.0415560 \
#~ 0.0556434 -0.2040259 1.0572252" \
#~ -set colorspace RGB ${15} $(printf "${10}/${11}_%06d.${12}" ${count})
#~ See http://www.imagemagick.org/discourse-server/viewtopic.php?t=21161
export -f img_par export -f img_par
@ -1583,30 +1679,25 @@ for ARG in "${FILE_ARGS_ITER[@]}"; do #Go through FILE_ARGS_ITER array, copied f
echo -e "\n" echo -e "\n"
#FFMPEG Filter Application: Temporal Denoising, 3D LUTs so far. See construction of $V_FILTERS in PREPARATION. #FFMPEG Filter Application: Temporal Denoising, 3D LUTs, Deshake, hqdn Denoising so far. See construction of $V_FILTERS in PREPARATION.
if [[ $FFMPEG_FILTERS == true ]]; then if [[ $FFMPEG_FILTERS == true ]]; then
tmpFiltered=${TMP}/filtered tmpFiltered=${TMP}/filtered
mkdir $tmpFiltered mkdir $tmpFiltered
#Give correct output. #Give correct output.
if [[ $isTemp == true && $isLUT == true ]]; then echo -e "\e[1mApplying Filters:\e[0m $(joinArgs ", " "${FILTER_ARR[@]}")...\n"
echo -e "\e[1m${TRUNC_ARG}:\e[0m Temporal Denoising and LUT Application...\n"
elif [[ $isTemp == false && $isLUT == true ]]; then
echo -e "\e[1m${TRUNC_ARG}:\e[0m LUT Application...\n"
elif [[ $isTemp == true && $isLUT == false ]]; then
echo -e "\e[1m${TRUNC_ARG}:\e[0m Temporal Denoising...\n"
fi
if [ $IMG_FMT == "exr" ]; then if [ $IMG_FMT == "exr" ]; then
echo -e "Note: EXRs may hang after filtering.\n" echo -e "Note: EXRs may hang after filtering.\n"
img_res=$(identify ${SEQ}/${TRUNC_ARG}_$(printf "%06d" $(echo "$FRAME_START" | bc)).${IMG_FMT} | cut -d$' ' -f3) img_res=$(identify ${SEQ}/${TRUNC_ARG}_$(printf "%06d" $(echo "$FRAME_START" | bc)).${IMG_FMT} | cut -d$' ' -f3)
convert -set colorspace RGB -define stream:buffer-size=0 -set colorspace RGB "${SEQ}/${TRUNC_ARG}_%06d.exr[${FRAME_START}-${FRAME_END}]" ppm:- | \ convert "${SEQ}/${TRUNC_ARG}_%06d.exr[${FRAME_START}-${FRAME_END}]" -define stream:buffer-size=0 -set colorspace RGB ppm:- | \
ffmpeg -f image2pipe -vcodec ppm -s "${img_res}" -r $FPS -loglevel panic -stats -i pipe:0 \ ffmpeg -f image2pipe -vcodec ppm -s "${img_res}" -r $FPS -loglevel panic -stats -i pipe:0 \
$V_FILTERS \ $V_FILTERS \
-vcodec ppm -n -r $FPS -f image2pipe pipe:1 | \ -vcodec ppm -n -r $FPS -f image2pipe pipe:1 | \
convert -depth 16 - -set colorspace RGB -compress piz -set colorspace RGB "${tmpFiltered}/%06d.${IMG_FMT}" #Watch colorspace. convert -depth 16 - -colorspace RGB -compress piz -set colorspace RGB "${tmpFiltered}/%06d.${IMG_FMT}"
#For some reason, this whole process sends EXR's into sRGB. That's why -colorspace RGB is specified. See Nasty Hacks.
else else
ffmpeg -start_number $FRAME_START -f image2 -i "${SEQ}/${TRUNC_ARG}_%06d.${IMG_FMT}" -loglevel panic -stats $V_FILTERS "${tmpFiltered}/%06d.${IMG_FMT}" ffmpeg -start_number $FRAME_START -f image2 -i "${SEQ}/${TRUNC_ARG}_%06d.${IMG_FMT}" -loglevel panic -stats $V_FILTERS "${tmpFiltered}/%06d.${IMG_FMT}"
fi fi
@ -1648,10 +1739,10 @@ for ARG in "${FILE_ARGS_ITER[@]}"; do #Go through FILE_ARGS_ITER array, copied f
V_FILTERS_PROX="-vf $FINAL_SCALE" V_FILTERS_PROX="-vf $FINAL_SCALE"
if [ $isH264 == true ]; then if [ $isH264 == true ]; then
echo -e "\e[1m${TRUNC_ARG}:\e[0m Encoding to ProRes/H.264..." echo -e "\e[1m${TRUNC_ARG}:\e[0m Encoding to ProRes/H.264..."
runSim dcrawImg mov_main mov_prox runSim dcrawOpt mov_main mov_prox #Used to run dcrawImg
else else
echo -e "\e[1m${TRUNC_ARG}:\e[0m Encoding to ProRes..." echo -e "\e[1m${TRUNC_ARG}:\e[0m Encoding to ProRes..."
dcrawImg | mov_main dcrawOpt | mov_main #Used to run dcrawImg
fi fi
echo "" echo ""
fi fi
@ -1660,11 +1751,13 @@ for ARG in "${FILE_ARGS_ITER[@]}"; do #Go through FILE_ARGS_ITER array, copied f
echo -e "\e[1m${TRUNC_ARG}:\e[0m Encoding to H.264..." echo -e "\e[1m${TRUNC_ARG}:\e[0m Encoding to H.264..."
if [ $IMAGES == true ]; then if [ $IMAGES == true ]; then
V_FILTERS_PROX="-vf $FINAL_SCALE" #See above note. V_FILTERS_PROX="-vf $FINAL_SCALE" #See above note.
dcrawImg | mov_prox dcrawOpt | mov_prox #Used to run dcrawImg
else else
dcrawOpt | mov_prox dcrawOpt | mov_prox
fi fi
echo "" echo ""
#Nasty Hack Part 3: We can no longer use dcrawImg; colorspace conversions are a fucking circus.
#Luckily, dcrawOpt isn't that much slower; even faster sometimes. Damn ppm conversion never pulled their weight.
fi fi
#Potentially move DNGs. #Potentially move DNGs.

3
sRange.py
View File

@ -37,6 +37,7 @@ def splitThreadRange(inNum, inThreads) :
if __name__ == "__main__" : if __name__ == "__main__" :
num = int(sys.argv[1]) num = int(sys.argv[1])
threads = int(sys.argv[2]) threads = int(sys.argv[2])
print(*splitThreadRange(num, threads)) if num == 1: print("0-0")
else: print(*splitThreadRange(num, threads))
#mlv_dump cannot use 0-0. #mlv_dump cannot use 0-0.