so-rose
/
convmlv
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 3 Wiki Activity

Huge update: Multithreaded DNG, new argument parsing & long arguments, reuse root folders, 

global config file, more error checking, use s and e in frame range, DNG progress bar, bug fixes galore!!
modularize
Sofus Albert Høgsbro Rose 2016-05-18 17:36:28 -04:00
parent 9aeea608eb
commit dfa4d226cf
3 changed files with 594 additions and 406 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

160
README.md
View File

@ -11,134 +11,146 @@ Help page is below:
```text ```text
Usage: Usage:
./convmlv.sh [OPTIONS] mlv_files ./convmlv.sh [FLAGS] [OPTIONS] files
INFO: INFO:
A script allowing you to convert .MLV, .RAW, or a folder with a DNG sequence into a sequence/movie with optional proxies. Images A script allowing you to develop ML files into workable formats. Many useful options are exposed.
are auto compressed. Many useful options are exposed, including formats (EXR by default). -->Image Defaults: Compressed 16-bit Linear EXR.
-->Acceptable Inputs: MLV, RAW, DNG Folder.
-->Option Input: From command line or config file.
VERSION: 1.8.2 VERSION: 1.9.0
DEPENDENCIES: If you don't use a feature, you don't need the dependency, though it's best to download them all. MANUAL DEPENDENCIES:
-mlv_dump: For DNG extraction from MLV. http://www.magiclantern.fm/forum/index.php?topic=7122.0 -mlv_dump: Required. http://www.magiclantern.fm/forum/index.php?topic=7122.0
-raw2dng: For DNG extraction from RAW. http://www.magiclantern.fm/forum/index.php?topic=5404.0 -raw2dng: For DNG extraction from RAW. http://www.magiclantern.fm/forum/index.php?topic=5404.0
-mlv2badpixels.sh: For bad pixel removal. https://bitbucket.org/daniel_fort/ml-focus-pixels/src -mlv2badpixels.sh: For bad pixel removal. https://bitbucket.org/daniel_fort/ml-focus-pixels/src
-dcraw: For RAW development. -cr2hdr: For Dual ISO Development. Two links: http://www.magiclantern.fm/forum/index.php?topic=16799.0
-ffmpeg: For video creation. -sRange.py: Required. See convmlv repository.
-ImageMagick: Used for making proxy sequence. -balance.py: For Auto White Balance. See convmlv repository.
-Python 3 + libs: Used for auto white balance.
-exiftool: Used in mlv2badpixels.sh.
OPTIONS, BASIC: OPTIONS, BASIC:
-v version - Print out version string. -v, --version version - Print out version string.
-o<path> OUTDIR - The path in which files will be placed (no space btwn -o and path). -h, --help help - Print out this help page.
-M<path> MLV_DUMP - The path to mlv_dump (no space btwn -M and path). Default is './mlv_dump'.
-R<path> RAW_DUMP - The path to raw2dng (no space btwn -M and path). Default is './raw2dng'.
-y<path> PYTHON - The path or command used to invoke Python. Defaults to python3.
-B<path> MLV_BP - The path to mlv2badpixels.sh (by dfort). Default is './mlv2badpixels.sh'.
-T[int] Max process threads, for multithreaded parts of the program. Defaults to 8. -C, --config CONFIG - Designates config file to use.
-o, --outdir <path> OUTDIR - The path in which files will be placed.
-P, --res-path <path> RES_PATH - The path in which all manual dependencies are looked for.
--mlv-dump <path> MLV_DUMP - The path to mlv_dump.
--raw-dump <path> RAW_DUMP - The path to raw2dng.
--badpixels <path> MLV_BP - The path to mlv2badpixels.sh (by dfort).
--cr-hdr <path> CR_HDR
--srange <path> SRANGE
--balance <path> BAL
--python <path> PYTHON - The path or command used to invoke Python.
-T, --threads [int] THREADS - Override amount of utilized process threads
OPTIONS, OUTPUT: OPTIONS, OUTPUT:
-i IMAGE - Specify to create an image sequence (EXR by default). -i IMAGE - Will output image sequence.
-f[0:3] IMG_FMT - Create a sequence of <format> format. 0 is default. -t [0:3] IMG_FMT - Specified image output format.
--> 0: EXR (default), 1: TIFF, 2: PNG, 3: Cineon (DPX)." --> 0: EXR (default), 1: TIFF, 2: PNG, 3: Cineon (DPX)."
-c COMPRESS - Specify to turn ***off*** automatic image compression. Auto compression options otherwise used: -m MOVIE - Will output a Prores4444 file.
--> TIFF: ZIP (best for 16-bit), PIZ for EXR (best for grainy images), PNG: lvl 9 (zlib deflate), DPX: RLE.
-m MOVIE - Specify to create a Prores4444 video. -p [0:3] PROXY - Create proxies alongside main output.
--> 0: No proxies (Default). 1: H.264 proxy. 2: JPG proxy sequence. 3: Both.
-p[0:3] PROXY - Specifies the proxy mode. 0 is default. --> JPG proxy *won't* be developed w/o IMAGE. H.264 proxy *will* be developed no matter what, if specified.
--> 0: No proxies. 1: H.264 proxy. 2: JPG proxy sequence. 3: Both.
--> JPG proxy won't be developed w/o -i. H.264 proxy will be developed no matter what, if specified.
-s [0%:100%] PROXY_SCALE - the size, in %, of the proxy output. -s [0%:100%] PROXY_SCALE - the size, in %, of the proxy output.
--> Use -s<percentage>% (no space). 50% is default. --> 50% is default.
-k KEEP_DNGS - Specify if you want to keep the DNG files. -k KEEP_DNGS - Specify if you want to keep the DNG files.
--> If you run convmlv on the dng_<name> folder, you will reuse those DNGs - no need to redevelop! --> Run convmlv on the top level folder of former output to reuse saved DNGs from that run!
-E<range> FRAME_RANGE - Specify to process only this frame range. -r <start>-<end> FRAME_RANGE - Specify to output this frame range only.
--> Use s and e appropriately to specify start and end. --> You may use s and e, such that s = start frame, e = end frame.
--> <range> must be written as <start>-<end>, indexed from 0 to (# of frames - 1). --> Indexed from 0 to (# of frames - 1).
--> If you write a single number, only that frame will be developed. --> A single number may be writted to develop that frame only.
--uncompress UNCOMP - Turns off lossless image compression. Otherwise:
--> TIFF: ZIP, EXR: PIZ, PNG: lvl 9 (zlib deflate), DPX: RLE.
OPTIONS, RAW DEVELOPMENT: OPTIONS, RAW DEVELOPMENT:
-d[0:3] DEMO_MODE - DCraw demosaicing mode. Higher modes are slower. 1 is default. -d [0:3] DEMO_MODE - Demosaicing algorithm. Higher modes are slower + better.
--> Use -d<mode> (no space). 0: Bilinear. 1: VNG (default). 2: PPG. 3: AHD. --> 0: Bilinear. 1: VNG (default). 2: PPG. 3: AHD.
-r FOUR_COLOR - Interpolate as four colors. Can often fix weirdness with VNG/AHD. -f FOUR_COLOR - Interpolate as RGBG. Can often fix weirdness with VNG/AHD.
-H[0:9] HIGHLIGHT_MODE - 2 looks the best, but can break. 0 is a safe bet. -H [0:9] HIGHLIGHT_MODE - Highlight management options.
--> Use -H<number> (no space). 0 clips. 1 allows colored highlights. 2 adjusts highlights to grey. --> 0: White, clipped highlights. 1: Clipped, colored highlights. 2: Similar to 1, but adjusted to grey.
--> 3 through 9 do highlight reconstruction with a certain tone. See dcraw documentation. --> 3-9: Highlight reconstruction. Can cause flickering; 1 or 2 usually give better results.
-C[0:3] CHROMA_SMOOTH - Apply chroma smoothing to the footage, which may help ex. with noise/bad pixels. -c [0:3] CHROMA_SMOOTH - Apply shadow/highlight chroma smoothing to the footage.
--> 0: None (default). 1: 2x2. 2: 3x3. 3: 5x5. --> 0: None (default). 1: 2x2. 2: 3x3. 3: 5x5.
--> Only applied to .MLV files. --> MLV Only.
-n[int] NOISE_REDUC - This is the threshold of wavelet denoising - specify to use. -n [int] NOISE_REDUC - Apply wavelet denoising.
--> Use -n<number>. Defaults to no denoising. 150 tends to be a good setting; 350 starts to look strange. --> Default: None. Subtle: 50. Medium: 100. Strong: 200.
-g[0:4] SPACE - This is output color space. 0 is default. -g [0:4] SPACE - Output color transformation.
--> Use -g<mode> (no space). 0: Linear. 1: 2.2 (Adobe RGB). 2: 1.8 (ProPhoto RGB). 3: sRGB. 4: BT.709. --> 0: Linear. 1: 2.2 (Adobe RGB). 2: 1.8 (ProPhoto RGB). 3: sRGB. 4: BT.709.
-S SHALLOW - Specifying this option will create an 8-bit output instead of a 16-bit output. --shallow SHALLOW - Output 8-bit files.
--> It'll kind of ruin the point of RAW, though....
OPTIONS, COLOR: OPTIONS, COLOR:
-w[0:2] WHITE - This is a modal white balance setting. Defaults to 1. -w [0:2] WHITE - This is a modal white balance setting.
--> Use -w<mode> (no space). --> 0: Auto WB. 1: Camera WB (default). 2: No Change.
--> 0: Auto WB (Requires Python Deps). 1: Camera WB. 2: No Change.
-L WHITE_SCALE - Specify to allow channels to clip as a result of any white balance. -l <path> LUT - Specify a LUT to apply.
--> Information loss occurs in certain situations. --> Supports cube, 3dl, dat, m3d.
--> LUT cannot be applied to EXR sequences.
-t[int] SATPOINT - Specify the 14-bit saturation point of your camera. -S [int] SATPOINT - Specify the 14-bit saturation point of your camera.
--> Lower if -H1 yields purple highlights. Must be correct for highlight reconstruction. --> Lower if -H1 yields purple highlights. Must be correct for highlight reconstruction.
--> Determine using the max value of 'dcraw -D -j -4 -T' --> Determine using the max value of 'dcraw -D -j -4 -T'
-A[int] WHITE_SPD - This is the amount of samples from which AWB will be calculated. --white-speed [int] WHITE_SPD - Samples used to calculate AWB
-->About this many frames, averaged over the course of the sequence, will be used to do AWB.
-l<path> LUT - This is a path to the 3D LUT. Specify the path to the LUT to use it. --allow-white-clip WHITE_SCALE - Let White Balance multipliers clip.
--> Compatibility determined by ffmpeg (.cube is supported).
--> LUT cannot be applied to EXR sequences.
--> Path to LUT (no space between -l and path).
OPTIONS, FEATURES: OPTIONS, FEATURES:
-u DUAL_ISO - Process file as dual ISO. -u DUAL_ISO - Process as dual ISO.
-b BADPIXELS - Fix focus pixels issue using dfort's script. -b BADPIXELS - Fix focus pixels issue using dfort's script.
--> His file can be found at https://bitbucket.org/daniel_fort/ml-focus-pixels/src.
-a<path> BADPIXEL_PATH - Use, appending to the generated one, your own .badpixels file. -a <path> BADPIXEL_PATH - Use your own .badpixels file.
--> Use -a<path> (no space). How to: http://www.dl-c.com/board/viewtopic.php?f=4&t=686 --> How to: http://www.dl-c.com/board/viewtopic.php?f=4&t=686
-F<path> DARKFRAME - This is the path to the dark frame MLV, for noise reduction. -F <path> DARKFRAME - This is the path to a "dark frame MLV"; effective for noise reduction.
--> This is a noise reduction technique: Record 5 sec w/lens cap on & same settings as footage. --> How to: Record 5 sec w/lens cap on & same settings as footage. Pass MLV in here.
--> Pass in that MLV file (not .RAW) as <path> to get noise reduction on all passed MLV files.
--> If the file extension is '.darkframe', the file will be used as the preaveraged dark frame. --> If the file extension is '.darkframe', the file will be used as the preaveraged dark frame.
-R <path> dark_out - Specify to create a .darkframe file from passed in MLV.
--> Outputs <arg>.darkframe file to <path>.
OPTIONS, INFO: OPTIONS, INFO:
-e Output MLV settings. -e Output MLV settings.
-K Debian Package Deps - Lists dependecies. Works with apt-get on Debian; should be similar elsewhere. -K Debian Package Deps - Output package dependecies.
--> No operations will be done. --> Install (Debian only): sudo apt-get install $ (./convmlv -K)
--> Example: sudo apt-get install $ (./convmlv -K)
-Y Python Deps - Lists Python dependencies. Works with pip. -Y Python Deps - Lists Python dependencies. Works directly with pip.
--> No operations will be done. -->Install (Linux): sudo pip3 install $ (./convmlv -Y)
--> Example: sudo pip3 install $ (./convmlv -Y)
-N Manual Deps - Lists manual dependencies, which must be downloaded by hand. -N Manual Deps - Lists manual dependencies, which must be downloaded by hand.
--> There's no automatic way to install these. See the forum post. --> There's no automatic way to install these. See http://www.magiclantern.fm/forum/index.php?topic=16799.0 .
CONFIG FILE:
Next to each option is an uppercased item, ex. OUTDIR. In a convmlv config file, you can specify this option
in the following format, line by line:
<VARNAME> <VALUE>
Some more notes regarding config files:
-The global config file, read every time, will be looked for in /home/sofus/convmlv.conf.
-Hashtags are considered comments, if and only if they are the first character in the line.
``` ```

700
convmlv.sh
View File

@ -1,14 +1,19 @@
#!/bin/bash #!/bin/bash
#TODO: #TODO:
#~ Stats for .RAW files and DNG sequences, best as possible. #~ Stats for .RAW files and naked DNG sequences, best as possible.
#~ --> Only read the file once into a long string, as opposed to once per setting. #~ --> Only read the file once into a long string, as opposed to once per setting.
#~ Read from Config File (update confEval).
#~ .darkframe generation.
#~ Better Preview: #~ Better Preview:
#~ --> Essentially, a different module (like -e) for seeing, not developing, footage. #~ --> Essentially, a different module (like -e) for seeing, not developing, footage.
#~ --> To start, an option allowing one to see a single frame, developed. -DONE #~ --> To start, an option allowing one to see a single frame, developed. -DONE
#BUG: Relative OUTDIR makes baxpixel generation fail if ./mlv2badpixels.sh doesn't exist. Fixed on Linux only.
#MAYBE FIXED BUG: Run on all Charleston files; determine what's making Black Level not appear sometimes. #MAYBE FIXED BUG: Run on all Charleston files; determine what's making Black Level not appear sometimes.
@ -41,7 +46,7 @@ if [[ $OSTYPE == "linux-gnu" ]]; then
else else
THREADS=4 THREADS=4
fi fi
#sysctl -n hw.ncpu for Mac #sysctl -n hw.ncpu for Mac?
#DEPENDENCIES #DEPENDENCIES
DEB_DEPS="imagemagick dcraw ffmpeg python3 python3-pip exiftool" #Dependency package names (Debian). List with -K option. DEB_DEPS="imagemagick dcraw ffmpeg python3 python3-pip exiftool" #Dependency package names (Debian). List with -K option.
@ -124,143 +129,150 @@ LUT=""
isLUT=false isLUT=false
help() { help() {
cat << EOF less -R << EOF
Usage: Usage:
$(echo -e "\033[1m./convmlv.sh\033[0m [OPTIONS] \033[2mmlv_files\033[0m") $(echo -e "\033[1m./convmlv.sh\033[0m [FLAGS] [OPTIONS] \033[2mfiles\033[0m")
INFO: INFO:
A script allowing you to convert .MLV, .RAW, or a folder with a DNG sequence into a sequence/movie with optional proxies. Images A script allowing you to develop ML files into workable formats. Many useful options are exposed.
are auto compressed. Many useful options are exposed, including formats (EXR by default). -->Image Defaults: Compressed 16-bit Linear EXR.
-->Acceptable Inputs: MLV, RAW, DNG Folder.
-->Option Input: From command line or config file.
$(echo -e "VERSION: ${VERSION}") $(echo -e "VERSION: ${VERSION}")
DEPENDENCIES: If you don't use a feature, you don't need the dependency, though it's best to download them all. MANUAL DEPENDENCIES:
-mlv_dump: For DNG extraction from MLV. http://www.magiclantern.fm/forum/index.php?topic=7122.0 -mlv_dump: Required. http://www.magiclantern.fm/forum/index.php?topic=7122.0
-raw2dng: For DNG extraction from RAW. http://www.magiclantern.fm/forum/index.php?topic=5404.0 -raw2dng: For DNG extraction from RAW. http://www.magiclantern.fm/forum/index.php?topic=5404.0
-mlv2badpixels.sh: For bad pixel removal. https://bitbucket.org/daniel_fort/ml-focus-pixels/src -mlv2badpixels.sh: For bad pixel removal. https://bitbucket.org/daniel_fort/ml-focus-pixels/src
-dcraw: For RAW development. -cr2hdr: For Dual ISO Development. Two links: http://www.magiclantern.fm/forum/index.php?topic=16799.0
-ffmpeg: For video creation. -sRange.py: Required. See convmlv repository.
-ImageMagick: Used for making proxy sequence. -balance.py: For Auto White Balance. See convmlv repository.
-Python 3 + libs: Used for auto white balance.
-exiftool: Used in mlv2badpixels.sh.
OPTIONS, BASIC: OPTIONS, BASIC:
-v version - Print out version string. -v, --version version - Print out version string.
-h help - Print out this help page. -h, --help help - Print out this help page.
-o<path> OUTDIR - The path in which files will be placed (no space btwn -o and path). -C, --config CONFIG - Designates config file to use.
-P<path> RES_PATH - The path in which all manual dependencies can be found. Will check ~/convmlv.conf, then the current directory.
--> In 'convmlv.conv', this can be specified using a line 'RES_PATH <path>'.
-M<path> MLV_DUMP - The path to mlv_dump. Default is '${RES_PATH}/mlv_dump'. -o, --outdir <path> OUTDIR - The path in which files will be placed.
-R<path> RAW_DUMP - The path to raw2dng. Default is '${RES_PATH}/raw2dng'. -P, --res-path <path> RES_PATH - The path in which all manual dependencies are looked for.
-y<path> PYTHON - The path or command used to invoke Python. Defaults to 'python3' on Linux, 'python' otherwise.
-B<path> MLV_BP - The path to mlv2badpixels.sh (by dfort). Default is '${RES_PATH}/mlv2badpixels.sh'.
-T[int] THREADS - Max process threads, for multithreaded parts of the program. Defaults to 8. --mlv-dump <path> MLV_DUMP - The path to mlv_dump.
--raw-dump <path> RAW_DUMP - The path to raw2dng.
--badpixels <path> MLV_BP - The path to mlv2badpixels.sh (by dfort).
--cr-hdr <path> CR_HDR
--srange <path> SRANGE
--balance <path> BAL
--python <path> PYTHON - The path or command used to invoke Python.
-T, --threads [int] THREADS - Override amount of utilized process threads
OPTIONS, OUTPUT: OPTIONS, OUTPUT:
-i IMAGE - Specify to create an image sequence (EXR by default). -i IMAGE - Will output image sequence.
-f[0:3] IMG_FMT - Create a sequence of <format> format. 0 is default. -t [0:3] IMG_FMT - Specified image output format.
--> 0: EXR (default), 1: TIFF, 2: PNG, 3: Cineon (DPX)." --> 0: EXR (default), 1: TIFF, 2: PNG, 3: Cineon (DPX)."
-c COMPRESS - Specify to turn ***off*** automatic image compression. Auto compression options otherwise used: -m MOVIE - Will output a Prores4444 file.
--> TIFF: ZIP (best for 16-bit), PIZ for EXR (best for grainy images), PNG: lvl 9 (zlib deflate), DPX: RLE.
-m MOVIE - Specify to create a Prores4444 video. -p [0:3] PROXY - Create proxies alongside main output.
--> 0: No proxies (Default). 1: H.264 proxy. 2: JPG proxy sequence. 3: Both.
-p[0:3] PROXY - Specifies the proxy mode. 0 is default. --> JPG proxy *won't* be developed w/o IMAGE. H.264 proxy *will* be developed no matter what, if specified.
--> 0: No proxies. 1: H.264 proxy. 2: JPG proxy sequence. 3: Both.
--> JPG proxy won't be developed w/o -i. H.264 proxy will be developed no matter what, if specified.
-s [0%:100%] PROXY_SCALE - the size, in %, of the proxy output. -s [0%:100%] PROXY_SCALE - the size, in %, of the proxy output.
--> Use -s<percentage>% (no space). 50% is default. --> 50% is default.
-k KEEP_DNGS - Specify if you want to keep the DNG files. -k KEEP_DNGS - Specify if you want to keep the DNG files.
--> If you run convmlv on the dng_<name> folder, you will reuse those DNGs - no need to redevelop! --> Run convmlv on the top level folder of former output to reuse saved DNGs from that run!
-E<range> FRAME_RANGE - Specify to process only this frame range. -r <start>-<end> FRAME_RANGE - Specify to output this frame range only.
--> Use s and e appropriately to specify start and end. --> You may use s and e, such that s = start frame, e = end frame.
--> <range> must be written as <start>-<end>, indexed from 0 to (# of frames - 1). --> Indexed from 0 to (# of frames - 1).
--> If you write a single number, only that frame will be developed. --> A single number may be writted to develop that frame only.
--uncompress UNCOMP - Turns off lossless image compression. Otherwise:
--> TIFF: ZIP, EXR: PIZ, PNG: lvl 9 (zlib deflate), DPX: RLE.
OPTIONS, RAW DEVELOPMENT: OPTIONS, RAW DEVELOPMENT:
-d[0:3] DEMO_MODE - DCraw demosaicing mode. Higher modes are slower. 1 is default. -d [0:3] DEMO_MODE - Demosaicing algorithm. Higher modes are slower + better.
--> Use -d<mode> (no space). 0: Bilinear. 1: VNG (default). 2: PPG. 3: AHD. --> 0: Bilinear. 1: VNG (default). 2: PPG. 3: AHD.
-r FOUR_COLOR - Interpolate as four colors. Can often fix weirdness with VNG/AHD. -f FOUR_COLOR - Interpolate as RGBG. Can often fix weirdness with VNG/AHD.
-H[0:9] HIGHLIGHT_MODE - 2 looks the best, but can break. 0 is a safe bet. -H [0:9] HIGHLIGHT_MODE - Highlight management options.
--> Use -H<number> (no space). 0 clips. 1 allows colored highlights. 2 adjusts highlights to grey. --> 0: White, clipped highlights. 1: Clipped, colored highlights. 2: Similar to 1, but adjusted to grey.
--> 3 through 9 do highlight reconstruction with a certain tone. See dcraw documentation. --> 3-9: Highlight reconstruction. Can cause flickering; 1 or 2 usually give better results.
-C[0:3] CHROMA_SMOOTH - Apply chroma smoothing to the footage, which may help ex. with noise/bad pixels. -c [0:3] CHROMA_SMOOTH - Apply shadow/highlight chroma smoothing to the footage.
--> 0: None (default). 1: 2x2. 2: 3x3. 3: 5x5. --> 0: None (default). 1: 2x2. 2: 3x3. 3: 5x5.
--> Only applied to .MLV files. --> MLV Only.
-n[int] NOISE_REDUC - This is the threshold of wavelet denoising - specify to use. -n [int] NOISE_REDUC - Apply wavelet denoising.
--> Use -n<number>. Defaults to no denoising. 150 tends to be a good setting; 350 starts to look strange. --> Default: None. Subtle: 50. Medium: 100. Strong: 200.
-g[0:4] SPACE - This is output color space. 0 is default. -g [0:4] SPACE - Output color transformation.
--> Use -g<mode> (no space). 0: Linear. 1: 2.2 (Adobe RGB). 2: 1.8 (ProPhoto RGB). 3: sRGB. 4: BT.709. --> 0: Linear. 1: 2.2 (Adobe RGB). 2: 1.8 (ProPhoto RGB). 3: sRGB. 4: BT.709.
-S SHALLOW - Specifying this option will create an 8-bit output instead of a 16-bit output. --shallow SHALLOW - Output 8-bit files.
--> It'll kind of ruin the point of RAW, though....
OPTIONS, COLOR: OPTIONS, COLOR:
-w[0:2] WHITE - This is a modal white balance setting. Defaults to 1. -w [0:2] WHITE - This is a modal white balance setting.
--> Use -w<mode> (no space). --> 0: Auto WB. 1: Camera WB (default). 2: No Change.
--> 0: Auto WB (Requires Python Deps). 1: Camera WB. 2: No Change.
-L WHITE_SCALE - Specify to allow channels to clip as a result of any white balance. -l <path> LUT - Specify a LUT to apply.
--> Information loss occurs in certain situations. --> Supports cube, 3dl, dat, m3d.
--> LUT cannot be applied to EXR sequences.
-t[int] SATPOINT - Specify the 14-bit saturation point of your camera. -S [int] SATPOINT - Specify the 14-bit saturation point of your camera.
--> Lower if -H1 yields purple highlights. Must be correct for highlight reconstruction. --> Lower if -H1 yields purple highlights. Must be correct for highlight reconstruction.
--> Determine using the max value of 'dcraw -D -j -4 -T' --> Determine using the max value of 'dcraw -D -j -4 -T'
-A[int] WHITE_SPD - This is the amount of samples from which AWB will be calculated. --white-speed [int] WHITE_SPD - Samples used to calculate AWB
-->About this many frames, averaged over the course of the sequence, will be used to do AWB.
-l<path> LUT - This is a path to the 3D LUT. Specify the path to the LUT to use it. --allow-white-clip WHITE_SCALE - Let White Balance multipliers clip.
--> Compatibility determined by ffmpeg (.cube is supported).
--> LUT cannot be applied to EXR sequences.
--> Path to LUT (no space between -l and path).
OPTIONS, FEATURES: OPTIONS, FEATURES:
-u DUAL_ISO - Process file as dual ISO. -u DUAL_ISO - Process as dual ISO.
-b BADPIXELS - Fix focus pixels issue using dfort's script. -b BADPIXELS - Fix focus pixels issue using dfort's script.
--> His file can be found at https://bitbucket.org/daniel_fort/ml-focus-pixels/src.
-a<path> BADPIXEL_PATH - Use, appending to the generated one, your own .badpixels file. -a <path> BADPIXEL_PATH - Use your own .badpixels file.
--> Use -a<path> (no space). How to: http://www.dl-c.com/board/viewtopic.php?f=4&t=686 --> How to: http://www.dl-c.com/board/viewtopic.php?f=4&t=686
-F<path> DARKFRAME - This is the path to the dark frame MLV, for noise reduction. -F <path> DARKFRAME - This is the path to a "dark frame MLV"; effective for noise reduction.
--> This is a noise reduction technique: Record 5 sec w/lens cap on & same settings as footage. --> How to: Record 5 sec w/lens cap on & same settings as footage. Pass MLV in here.
--> Pass in that MLV file (not .RAW) as <path> to get noise reduction on all passed MLV files.
--> If the file extension is '.darkframe', the file will be used as the preaveraged dark frame. --> If the file extension is '.darkframe', the file will be used as the preaveraged dark frame.
-R <path> dark_out - Specify to create a .darkframe file from passed in MLV.
--> Outputs <arg>.darkframe file to <path>.
OPTIONS, INFO: OPTIONS, INFO:
-e Output MLV settings. -e Output MLV settings.
-K Debian Package Deps - Lists dependecies. Works with apt-get on Debian; should be similar elsewhere. -K Debian Package Deps - Output package dependecies.
--> No operations will be done. --> Install (Debian only): sudo apt-get install $ (./convmlv -K)
--> Example: sudo apt-get install $ (./convmlv -K)
-Y Python Deps - Lists Python dependencies. Works with pip. -Y Python Deps - Lists Python dependencies. Works directly with pip.
--> No operations will be done. -->Install (Linux): sudo pip3 install $ (./convmlv -Y)
--> Example: sudo pip3 install $ (./convmlv -Y)
-N Manual Deps - Lists manual dependencies, which must be downloaded by hand. -N Manual Deps - Lists manual dependencies, which must be downloaded by hand.
--> There's no automatic way to install these. See the forum post. --> There's no automatic way to install these. See http://www.magiclantern.fm/forum/index.php?topic=16799.0 .
CONFIG FILE:
Next to each option is an uppercased item, ex. OUTDIR. In a convmlv config file, you can specify this option
in the following format, line by line:
<VARNAME> <VALUE>
Some more notes regarding config files:
-The global config file, read every time, will be looked for in $HOME/convmlv.conf.
-Hashtags are considered comments, if and only if they are the first character in the line.
EOF EOF
} }
@ -272,12 +284,14 @@ mkdirS() {
if [ -d $path ]; then if [ -d $path ]; then
while true; do while true; do
read -p "Overwrite ${path}? [y/n] " yn read -p "Overwrite ${path}? [y/n/q] " ynq
case $yn in case $ynq in
[Yy]* ) echo -e ""; rm -rf $path; mkdir -p $path >/dev/null 2>/dev/null; break [Yy]* ) echo -e ""; rm -rf $path; mkdir -p $path >/dev/null 2>/dev/null; break
;; ;;
[Nn]* ) echo -e "\n\e[0;31m\e[1mDirectory ${path} won't be created.\e[0m\n"; cont=true; `$cleanup`; break [Nn]* ) echo -e "\n\e[0;31m\e[1mDirectory ${path} won't be created.\e[0m\n"; cont=true; `$cleanup`; break
;; ;;
[Qq]* ) echo -e "\n\e[0;31m\e[1mHalting execution. Directory ${path} won't be created.\e[0m\n"; exit 1;
;;
* ) echo -e "\e[0;31m\e[1mPlease answer yes or no.\e[0m\n" * ) echo -e "\e[0;31m\e[1mPlease answer yes or no.\e[0m\n"
;; ;;
esac esac
@ -313,53 +327,125 @@ evalConf() { #Run BEFORE argument parsing.
;; ;;
esac esac
done < "${HOME}/convmlv.conf" done < "${HOME}/convmlv.conf"
#~ echo "$OUTDIR"
} }
parseArgs() { #Fixing this would be difficult. parseArgs() { #Amazing new argument parsing!!!
if [ ${ARG} == "-e" ]; then #This very special arguments would fuck everything up if left to roam free... longArg() { #Creates VAL
SETTINGS_OUTPUT=true ret="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
let ARGNUM-- }
continue while getopts "vh C o:P: T: i t: m p: s: k r: d: f H: c: n: g: w: l: S: u b a: F: e K Y N -:" opt; do
fi case "$opt" in
if [ `echo "${ARG}" | cut -c1-1` = "-" ]; then -) #Long Arguments
if [ `echo "${ARG}" | cut -c2-2` = "H" ]; then # case ${OPTARG} in
HIGHLIGHT_MODE=`echo "${ARG}" | cut -c3-3` outdir)
let ARGNUM-- val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
fi OUTDIR=$val
if [ `echo "${ARG}" | cut -c2-2` = "s" ]; then # ;;
PROXY_SCALE=`echo "${ARG}" | cut -c3-${#ARG} >/dev/null 2>/dev/null` #Might error. We'll check, no worries. version)
if [ -z $PROXY_SCALE ]; then echo -e "convmlv v${VERSION}"
echo -e "\e[0;31m\e[1mNo proxy scale set!\e[0m\n" ;;
exit 1 help)
fi help
let ARGNUM-- exit 0
fi ;;
if [ `echo ${ARG} | cut -c2-2` = "u" ]; then # config)
DUAL_ISO=true echo "To Be Implemented!"
let ARGNUM-- ;;
fi res-path)
if [ `echo ${ARG} | cut -c2-2` = "E" ]; then # val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
RANGE_BASE=$(echo ${ARG} | cut -c3-${#ARG}) RES_PATH=$val
setPaths #Set all the paths with the new RES_PATH.
;;
mlv-dump)
val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
MLV_DUMP=$val
;;
raw-dump)
val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
RAW_DUMP=$val
;;
badpixels)
val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
MLV_BP=$val
;;
cr-hdr)
val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
CR_HDR=$val
;;
srange)
val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
PYTHON_BAL=$val
setPaths #Must regen BAL
;;
balance)
val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
PYTHON_SRANGE=$val
setPaths #Must regen SRANGE
;;
python)
val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
PYTHON=$val
setPaths #Set all the paths with the new PYTHON.
;;
threads)
val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
THREADS=$val
;;
isFR=false
let ARGNUM-- uncompress)
fi isCOMPRESS=false
if [ `echo ${ARG} | cut -c2-2` = "F" ]; then # ;;
DARKFRAME=`echo ${ARG} | cut -c3-${#ARG}`
let ARGNUM--
fi shallow)
if [ `echo ${ARG} | cut -c2-2` = "t" ]; then DEPTH=""
tmpSat=`echo ${ARG} | cut -c3-${#ARG}` DEPTH_OUT="-depth 8"
SATPOINT="-S ${tmpSat}" ;;
let ARGNUM--
fi
if [ `echo ${ARG} | cut -c2-2` = "r" ]; then white-speed)
FOUR_COLOR="-f" val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
let ARGNUM-- WHITE_SPD=$val
fi ;;
if [ `echo ${ARG} | cut -c2-2` = "f" ]; then allow-white-clip)
mode=`echo ${ARG} | cut -c3-3` isScale=true
;;
*)
echo "Invalid option: -$OPTARG" >&2
;;
esac
;;
v)
echo -e "convmlv v${VERSION}"
;;
h)
help
exit 0
;;
C)
echo "To Be Implemented!"
;;
o)
OUTDIR=${OPTARG}
;;
P)
RES_PATH=${OPTARG}
setPaths #Set all the paths with the new RES_PATH.
;;
T)
THREADS=${OPTARG}
;;
i)
IMAGES=true
;;
t)
mode=${OPTARG}
case ${mode} in case ${mode} in
"0") IMG_FMT="exr" "0") IMG_FMT="exr"
;; ;;
@ -370,60 +456,12 @@ parseArgs() { #Fixing this would be difficult.
"3") IMG_FMT="dpx" "3") IMG_FMT="dpx"
;; ;;
esac esac
let ARGNUM--
fi
if [ `echo ${ARG} | cut -c2-2` = "C" ]; then
mode=`echo ${ARG} | cut -c3-3`
case ${mode} in
"0") CHROMA_SMOOTH="--no-cs"
;; ;;
"1") CHROMA_SMOOTH="--cs2x2" m)
;;
"2") CHROMA_SMOOTH="--cs3x3"
;;
"3") CHROMA_SMOOTH="--cs5x5"
;;
esac
let ARGNUM--
fi
if [ `echo ${ARG} | cut -c2-2` = "y" ]; then
PYTHON=`echo ${ARG} | cut -c3-${#ARG}`
setPaths #Set all the paths with the new PYTHON.
let ARGNUM--
fi
if [ `echo ${ARG} | cut -c2-2` = "v" ]; then
echo -e "convmlv v${VERSION}"
let ARGNUM--
fi
if [ `echo ${ARG} | cut -c2-2` = "m" ]; then
MOVIE=true MOVIE=true
let ARGNUM-- ;;
fi p)
if [ `echo ${ARG} | cut -c2-2` = "c" ]; then PROXY=${OPTARG}
isCOMPRESS=false
let ARGNUM--
fi
if [ `echo ${ARG} | cut -c2-2` = "L" ]; then
isScale=true
let ARGNUM--
fi
if [ `echo ${ARG} | cut -c2-2` = "M" ]; then
MLV_DUMP=`echo ${ARG} | cut -c3-${#ARG}`
let ARGNUM--
fi
if [ `echo ${ARG} | cut -c2-2` = "P" ]; then
RES_PATH=`echo ${ARG} | cut -c3-${#ARG}`
setPaths #Set all the paths with the new RES_PATH.
let ARGNUM--
fi
if [ `echo ${ARG} | cut -c2-2` = "R" ]; then
RAW_DUMP=`echo ${ARG} | cut -c3-${#ARG}`
let ARGNUM--
fi
if [ `echo ${ARG} | cut -c2-2` = "p" ]; then
PROXY=`echo ${ARG} | cut -c3-3`
case ${PROXY} in case ${PROXY} in
"0") isJPG=false; isH264=false "0") isJPG=false; isH264=false
;; ;;
@ -434,62 +472,73 @@ parseArgs() { #Fixing this would be difficult.
"3") isJPG=true; isH264=true "3") isJPG=true; isH264=true
;; ;;
esac esac
let ARGNUM-- ;;
fi s)
if [ `echo ${ARG} | cut -c2-2` = "i" ]; then PROXY_SCALE=${OPTARG}
IMAGES=true ;;
let ARGNUM-- k)
fi KEEP_DNGS=true
if [ `echo ${ARG} | cut -c2-2` = "o" ]; then ;;
OUTDIR=`echo ${ARG} | cut -c3-${#ARG}` r)
let ARGNUM-- RANGE_BASE=${OPTARG}
fi isFR=false
if [ `echo ${ARG} | cut -c2-2` = "a" ]; then ;;
BADPIXEL_PATH=`echo ${ARG} | cut -c3-${#ARG}`
let ARGNUM--
fi d)
if [ `echo ${ARG} | cut -c2-2` = "n" ]; then DEMO_MODE=${OPTARG}
setting=`echo ${ARG} | cut -c3-${#ARG}` ;;
NOISE_REDUC="-n ${setting}" f)
let ARGNUM-- FOUR_COLOR="-f"
fi ;;
if [ `echo ${ARG} | cut -c2-2` = "T" ]; then H)
setting=`echo ${ARG} | cut -c3-${#ARG}` HIGHLIGHT_MODE=${OPTARG}
THREADS=$setting ;;
let ARGNUM-- c)
fi mode=${OPTARG}
if [ `echo ${ARG} | cut -c2-2` = "h" ]; then
help
exit 0
fi
if [ `echo ${ARG} | cut -c2-2` = "d" ]; then
DEMO_MODE=`echo ${ARG} | cut -c3-3`
let ARGNUM--
fi
if [ `echo ${ARG} | cut -c2-2` = "g" ]; then
mode=`echo ${ARG} | cut -c3-3`
case ${mode} in case ${mode} in
"0") GAMMA="1 1"; SPACE="0" "0") CHROMA_SMOOTH="--no-cs"
;; ;;
"1") GAMMA="2.2 0"; SPACE="2" "1") CHROMA_SMOOTH="--cs2x2"
;; ;;
"2") GAMMA="1.8 0"; SPACE="4" "2") CHROMA_SMOOTH="--cs3x3"
;; ;;
"3") GAMMA="2.4 12.9"; SPACE="1" "3") CHROMA_SMOOTH="--cs5x5"
;;
"4") GAMMA="2.222 4.5"; SPACE="0"
;; ;;
esac esac
;;
n)
NOISE_REDUC="-n ${OPTARG}"
;;
g)
mode=${OPTARG}
case ${mode} in
"0")
GAMMA="1 1"
#~ SPACE="0" #What's going on here?
;;
"1")
GAMMA="2.2 0"
#~ SPACE="2"
;;
"2")
GAMMA="1.8 0"
#~ SPACE="4"
;;
"3")
GAMMA="2.4 12.9"
#~ SPACE="1"
;;
"4")
GAMMA="2.222 4.5"
#~ SPACE="0"
;;
esac
;;
let ARGNUM--
fi w)
if [ `echo ${ARG} | cut -c2-2` = "S" ]; then mode=${OPTARG}
DEPTH=""
DEPTH_OUT=""
let ARGNUM--
fi
if [ `echo ${ARG} | cut -c2-2` = "w" ]; then
mode=`echo ${ARG} | cut -c3-3`
case ${mode} in case ${mode} in
"0") CAMERA_WB=false; GEN_WHITE=true #Will generate white balance. "0") CAMERA_WB=false; GEN_WHITE=true #Will generate white balance.
;; ;;
@ -498,15 +547,9 @@ parseArgs() { #Fixing this would be difficult.
"2") WHITE="-r 1 1 1 1"; CAMERA_WB=false; GEN_WHITE=false "2") WHITE="-r 1 1 1 1"; CAMERA_WB=false; GEN_WHITE=false
;; ;;
esac esac
;;
let ARGNUM-- l)
fi LUT_PATH=${OPTARG}
if [ `echo ${ARG} | cut -c2-2` = "K" ]; then
echo $DEB_DEPS
exit 0
fi
if [ `echo ${ARG} | cut -c2-2` = "l" ]; then
LUT_PATH=`echo ${ARG} | cut -c3-${#ARG}`
if [ ! -f $LUT_PATH ]; then if [ ! -f $LUT_PATH ]; then
echo "LUT not found!!!" echo "LUT not found!!!"
echo $LUT_PATH echo $LUT_PATH
@ -514,47 +557,79 @@ parseArgs() { #Fixing this would be difficult.
fi fi
LUT="lut3d=${LUT_PATH}" LUT="lut3d=${LUT_PATH}"
isLUT=true isLUT=true
let ARGNUM-- ;;
fi S)
if [ `echo ${ARG} | cut -c2-2` = "b" ]; then tmpSat=${OPTARG}
SATPOINT="-S ${tmpSat}"
;;
u)
DUAL_ISO=true
;;
b)
isBP=true isBP=true
let ARGNUM-- ;;
fi a)
if [ `echo ${ARG} | cut -c2-2` = "k" ]; then BADPIXEL_PATH=${OPTARG}
KEEP_DNGS=true ;;
let ARGNUM-- F)
fi DARKFRAME=${OPTARG}
if [ `echo ${ARG} | cut -c2-2` = "B" ]; then ;;
MLV_BP=`echo ${ARG} | cut -c3-${#ARG}` R)
let ARGNUM-- echo "To Be Implemented"
fi ;;
if [ `echo ${ARG} | cut -c2-2` = "A" ]; then
WHITE_SPD=`echo ${ARG} | cut -c3-${#ARG}`
let ARGNUM-- e)
fi SETTINGS_OUTPUT=true
if [ `echo ${ARG} | cut -c2-2` = "Y" ]; then ;;
K)
echo $DEB_DEPS
;;
Y)
echo $PIP_DEPS echo $PIP_DEPS
exit 0 exit 0
fi ;;
if [ `echo ${ARG} | cut -c2-2` = "N" ]; then N)
echo $MAN_DEPS echo $MAN_DEPS
exit 0 exit 0
fi ;;
continue
fi
*)
echo "Invalid option: -$OPTARG" >&2
;;
esac
done
} }
checkDeps() { checkDeps() {
argBase="$(basename "$ARG")"
argExt="${argBase##*.}"
argTrunc="${argBase%.*}"
#Argument Checks
if [ ! -f $ARG ] && [ ! -d $ARG ]; then if [ ! -f $ARG ] && [ ! -d $ARG ]; then
echo -e "\e[0;31m\e[1mFile ${ARG} not found! Skipping file.\e[0m\n" echo -e "\e[0;31m\e[1mFile ${ARG} not found! Skipping file.\e[0m\n"
continue let ARGNUM--; continue
fi
if [[ ! -d $ARG && ! ( $argExt == "MLV" || $argExt == "mlv" || $argExt == "RAW" || $argExt == "raw" ) ]]; then
echo -e "\e[0;31m\e[1mFile ${ARG} has invalid extension!\e[0m\n"
let ARGNUM--; continue
fi
if [[ ( ( ! -f $ARG ) && $(ls -1 ${ARG%/}/*.[Dd][Nn][Gg] 2>/dev/null | wc -l) == 0 ) && ( `folderName ${ARG}` != $argTrunc ) ]]; then
echo -e "\e[0;31m\e[1mFolder ${ARG} contains no DNG files!\e[0m\n"
let ARGNUM--; continue
fi fi
if [ ! -d $ARG ] && [ $(echo $(wc -c ${ARG} | cut -d " " -f1) / 1000 | bc) -lt 1000 ]; then #Check that the file is not too small. if [ ! -d $ARG ] && [ $(echo $(wc -c ${ARG} | cut -d " " -f1) / 1000 | bc) -lt 1000 ]; then #Check that the file is not too small.
cont=false cont=false
while true; do while true; do
read -p "${ARG} is unusually small at $(wc -c ${ARG})KB. Continue, skip, or remove? [c/s/r] " csr read -p "${ARG} is unusually small at $(echo "$(echo "$(wc -c ${ARG})" | cut -d$' ' -f1) / 1000" | bc)KB. Continue, skip, or remove? [c/s/r] " csr
case $ysr in case $csr in
[Cc]* ) "\n\e[0;31m\e[1mContinuing.\e[0m\n"; break [Cc]* ) "\n\e[0;31m\e[1mContinuing.\e[0m\n"; break
;; ;;
[Ss]* ) echo -e "\n\e[0;31m\e[1mSkipping.\e[0m\n"; cont=true; break [Ss]* ) echo -e "\n\e[0;31m\e[1mSkipping.\e[0m\n"; cont=true; break
@ -567,7 +642,7 @@ checkDeps() {
done done
if [ $cont == true ]; then if [ $cont == true ]; then
continue let ARGNUM--; continue
fi fi
fi fi
@ -587,6 +662,8 @@ checkDeps() {
isExit=true isExit=true
fi fi
#Features
if [ ! -f $PYTHON_BAL ]; then if [ ! -f $PYTHON_BAL ]; then
echo -e "\e[0;31m\e[1m${PYTHON_BAL} not found! Execution will continue without AWB.\e[0m\n\tDownload from convmlv repository.\n" echo -e "\e[0;31m\e[1m${PYTHON_BAL} not found! Execution will continue without AWB.\e[0m\n\tDownload from convmlv repository.\n"
fi fi
@ -603,6 +680,7 @@ checkDeps() {
echo -e "\e[0;31m\e[1m${CR_HDR} not found! Execution will continue without Dual ISO processing capability.\e[0m\n\tGet it here: http://www.magiclantern.fm/forum/index.php?topic=7139.0\n" echo -e "\e[0;31m\e[1m${CR_HDR} not found! Execution will continue without Dual ISO processing capability.\e[0m\n\tGet it here: http://www.magiclantern.fm/forum/index.php?topic=7139.0\n"
fi fi
if [[ $isExit == true ]]; then if [[ $isExit == true ]]; then
echo -e "\e[0;33m\e[1mPlace all downloaded files in the Current Directory, or specify paths with relevant arguments (see 'convmlv -h')!\e[0m\n" echo -e "\e[0;33m\e[1mPlace all downloaded files in the Current Directory, or specify paths with relevant arguments (see 'convmlv -h')!\e[0m\n"
exit 1 exit 1
@ -613,6 +691,11 @@ bold() {
echo -e "\e[1m${1}\e[0m" echo -e "\e[1m${1}\e[0m"
} }
folderName() {
#Like basename, but for folders.
echo "$1" | rev | cut -d$'/' -f1 | rev
}
prntSet() { prntSet() {
cat << EOF cat << EOF
$(bold CameraName): ${CAM_NAME} $(bold CameraName): ${CAM_NAME}
@ -648,17 +731,39 @@ mlvSet() {
KELVIN=`echo "$camDump" | grep 'Kelvin' | sed 's/[[:alpha:] ]*: //' | cut -d$'\n' -f1` KELVIN=`echo "$camDump" | grep 'Kelvin' | sed 's/[[:alpha:] ]*: //' | cut -d$'\n' -f1`
} }
rawSet() { #Set as many options as the RAW spec will allow. Grey out the rest.
#~ camDump=$(${MLV_DUMP} -v -m ${ARG}) #Read it in *once*; otherwise it's unbearably slow on external media.
#~ FPS=`echo "$camDump" | grep FPS | awk 'FNR == 1 {print $3}'`
#~ CAM_NAME=`echo "$camDump" | grep 'Camera Name' | cut -d "'" -f 2`
#~ FRAMES=`echo "$camDump" | awk '/Processed/ { print $2; }'` #Use actual processed frames as opposed to what the sometimes incorrect metadata thinks.
#~ ISO=`echo "$camDump" | grep 'ISO' | sed 's/[[:alpha:] ]*: //' | cut -d$'\n' -f2`
#~ APERTURE=`echo "$camDump" | grep 'Aperture' | sed 's/[[:alpha:] ]*: //' | cut -d$'\n' -f1`
#~ LEN_FOCAL=`echo "$camDump" | grep 'Focal Len' | sed 's/[[:alpha:] ]*: //' | cut -d$'\n' -f1`
#~ SHUTTER=`echo "$camDump" | grep 'Shutter' | sed 's/[[:alpha:] ]*: //' | grep -oP '\(\K[^)]+' | cut -d$'\n' -f1`
#~ REC_DATE=`echo "$camDump" | grep 'Date' | sed 's/[[:alpha:] ]*: //' | cut -d$'\n' -f1`
#~ REC_TIME=`echo "$camDump" | grep 'Time: [0-2][0-9]\:*' | sed 's/[[:alpha:] ]*: //' | cut -d$'\n' -f1`
#~ KELVIN=`echo "$camDump" | grep 'Kelvin' | sed 's/[[:alpha:] ]*: //' | cut -d$'\n' -f1`
echo ''
}
if [ $# == 0 ]; then if [ $# == 0 ]; then
help echo -e "\e[0;31m\e[1mNo arguments given.\e[0m\n\tType 'convmlv -h/--help' to see help page, or 'convmlv -v/--version' for current version string."
echo -e "\e[0;31m\e[1mNo arguments, no joy!!!\e[0m\n"
fi fi
parseArgs "$@" #Parse all arguments.
shift $((OPTIND-1))
ARGNUM=$# ARGNUM=$#
for ARG in $*; do
for ARG in $@; do #All remaining mass-arguments
ARG="$(pwd)/${ARG}"
if [[ $OSTYPE == "linux-gnu" ]]; then
ARG="$(readlink -f $ARG)" >/dev/null 2>/dev/null #Relative ARG only fixed on Linux, as readlink only exists in UNIX. Mac variant?
fi
#Evaluate convmlv.conf configuration file. Very limited for now. #Evaluate convmlv.conf configuration file. Very limited for now.
evalConf evalConf
#Evaluate command line arguments. ARGNUM decrements to keep track of how many files there are to process.
parseArgs # <-- Has a continue statement inside of it if we haven't reached the output.
#Check that things exist. #Check that things exist.
checkDeps checkDeps
@ -704,6 +809,10 @@ for ARG in $*; do
#PREPARATION #PREPARATION
#Establish Basic Directory Structure. #Establish Basic Directory Structure.
if [[ $OSTYPE == "linux-gnu" ]]; then
OUTDIR="$(readlink -f $OUTDIR)" >/dev/null 2>/dev/null #Relative Badpixel OUTDIR only fixed on Linux, as readlink only exists in UNIX. Mac variant?
fi
if [ $OUTDIR != $PWD ] && [ isOutGen == false ]; then if [ $OUTDIR != $PWD ] && [ isOutGen == false ]; then
mkdir -p $OUTDIR #NO RISKS. WE REMEMBER THE LUT.py. RIP. mkdir -p $OUTDIR #NO RISKS. WE REMEMBER THE LUT.py. RIP.
isOutGen=true isOutGen=true
@ -712,11 +821,21 @@ for ARG in $*; do
FILE="${OUTDIR}/${TRUNC_ARG}" FILE="${OUTDIR}/${TRUNC_ARG}"
TMP="${FILE}/tmp_${TRUNC_ARG}" TMP="${FILE}/tmp_${TRUNC_ARG}"
#Manage if it's a DNG argument. Also, create FILE and TMP. #Manage if it's a DNG argument, reused or not. Also, create FILE and TMP.
DEVELOP=true DEVELOP=true
if [ -d $ARG ] && [ `basename ${ARG} | cut -c1-3` == "dng" ] && [ -f "${ARG}/../settings.txt" ]; then #If we're reusing a dng sequence, copy over before we delete the original. if [[ ( -d $ARG ) && ( ( `basename ${ARG} | cut -c1-3` == "dng" && -f "${ARG}/../settings.txt" ) || ( `basename ${ARG}` == $TRUNC_ARG && -f "${ARG}/settings.txt" ) ) ]]; then #If we're reusing a dng sequence, copy over before we delete the original.
echo -e "\e[1m${TRUNC_ARG}:\e[0m Moving DNGs from previous run...\n" #Use prespecified DNG sequence. echo -e "\e[1m${TRUNC_ARG}:\e[0m Moving DNGs from previous run...\n" #Use prespecified DNG sequence.
#User may specify either the dng_ or the trunc_arg folder; must account for both.
if [[ `folderName ${ARG}` == $TRUNC_ARG && -d "${ARG}/dng_${TRUNC_ARG}" ]]; then
ARG="${ARG}/dng_${TRUNC_ARG}" #Set arg to the dng argument.
elif [[ `folderName ${ARG}` == $TRUNC_ARG ]]; then
echo -e "\e[0;31m\e[1mCannot reuse - DNG folder does not exist! Skipping argument.\e[0m"
continue
else
TRUNC_ARG=`echo $TRUNC_ARG | cut -c5-${#TRUNC_ARG}`
fi
DNG_LOC=${OUTDIR}/tmp_reused DNG_LOC=${OUTDIR}/tmp_reused
mkdir -p ${OUTDIR}/tmp_reused mkdir -p ${OUTDIR}/tmp_reused
@ -726,7 +845,6 @@ for ARG in $*; do
FRAMES=`cat ${ARG}/../settings.txt | grep "Frames" | cut -d $" " -f2` #Grab FRAMES from previous run. FRAMES=`cat ${ARG}/../settings.txt | grep "Frames" | cut -d $" " -f2` #Grab FRAMES from previous run.
cp "${ARG}/../settings.txt" $DNG_LOC cp "${ARG}/../settings.txt" $DNG_LOC
TRUNC_ARG=`echo $TRUNC_ARG | cut -c5-${#TRUNC_ARG}`
oldARG=$ARG oldARG=$ARG
ARG=$(dirname $ARG)/${TRUNC_ARG} ARG=$(dirname $ARG)/${TRUNC_ARG}
BASE="$(basename "$ARG")" BASE="$(basename "$ARG")"
@ -813,7 +931,7 @@ for ARG in $*; do
rawStat="\c" rawStat="\c"
fi fi
#IF extension is RAW, convert to MLV. All the newer features are MLV-only, because of mlv_dump's amazingness. #IF extension is RAW, we want to convert to MLV. All the newer features are MLV-only, because of mlv_dump's amazingness.
if [ $EXT == "MLV" ] || [ $EXT == "mlv" ]; then if [ $EXT == "MLV" ] || [ $EXT == "mlv" ]; then
# Read the header for interesting settings :) . # Read the header for interesting settings :) .
@ -843,13 +961,20 @@ for ARG in $*; do
#Looks like this: 0-1 2-2 3-4 5-5 6-7 8-8 9-10. Put that in an array. #Looks like this: 0-1 2-2 3-4 5-5 6-7 8-8 9-10. Put that in an array.
devDNG() { #Takes n arguments: 1{}, the frame range 2$MLV_DUMP 3$REAL_MLV 4$DARK_PROC 5$tmpOut 6$smooth 7$TMP 8$FRAME_END 9$TRUNC_ARG devDNG() { #Takes n arguments: 1{}, the frame range 2$MLV_DUMP 3$REAL_MLV 4$DARK_PROC 5$tmpOut 6$smooth 7$TMP 8$FRAME_END 9$TRUNC_ARG
tmpOut=${7}/${1} #Each output will number from 0, so give each its own folder. range=$1
firstFrame=false
if [[ $range == "0-0" ]]; then #mlv_dump can't handle 0-0, so we develop 0-1.
range="0-1"
firstFrame=true
fi
tmpOut=${7}/${range} #Each output will number from 0, so give each its own folder.
mkdir -p $tmpOut mkdir -p $tmpOut
start=$(echo "$1" | cut -d'-' -f1) start=$(echo "$range" | cut -d'-' -f1)
end=$(echo "$1" | cut -d'-' -f2) #Get start and end frames from the frame range end=$(echo "$range" | cut -d'-' -f2) #Get start and end frames from the frame range
$2 $3 $4 -o "${tmpOut}/${9}_" -f ${1} $6 --dng --batch | { #mlv_dump command. Uses frame range. $2 $3 $4 -o "${tmpOut}/${9}_" -f ${range} $6 --dng --batch | { #mlv_dump command. Uses frame range.
lastCur=0 lastCur=0
while IFS= read -r line; do while IFS= read -r line; do
output=$(echo $line | grep -Po 'V.*A' | cut -d':' -f2 | cut -d$' ' -f1) #Hacked my way to the important bit. output=$(echo $line | grep -Po 'V.*A' | cut -d':' -f2 | cut -d$' ' -f1) #Hacked my way to the important bit.
@ -863,6 +988,11 @@ for ARG in $*; do
done done
} #Progress Bar } #Progress Bar
if [[ $firstFrame == true ]]; then #If 0-0.
rm $(printf "${tmpOut}/${9}_%06d.dng" 1) #Remove frame #1, leaving us only with frame #0.
mv $tmpOut "${7}/0-0" #Move back to 0-0, as if that's how it was developed all along.
fi
} }
export -f devDNG #Export to run in subshell. export -f devDNG #Export to run in subshell.
@ -871,9 +1001,9 @@ for ARG in $*; do
xargs -I {} -P $THREADS -n 1 \ xargs -I {} -P $THREADS -n 1 \
bash -c "devDNG '{}' '$MLV_DUMP' '$REAL_MLV' '$DARK_PROC' '$tmpOut' '$smooth' '$TMP' '$FRAME_END' '$TRUNC_ARG'" bash -c "devDNG '{}' '$MLV_DUMP' '$REAL_MLV' '$DARK_PROC' '$tmpOut' '$smooth' '$TMP' '$FRAME_END' '$TRUNC_ARG'"
#Since devDNG must run in a subshell, globals don't follow. Must pass *everything* in. #Since devDNG must run in a subshell, globals don't follow. Must pass *everything* in.
echo -e "\e[2K\rMLV to DNG: Frame ${FRAME_END}/${FRAME_END}\c" #Ensure it looks right at the end. echo -e "\e[2K\rMLV to DNG: Frame ${FRAME_END}/${FRAME_END}\c" #Ensure it looks right at the end.
echo -e "\n" echo -e "\n"
#~ exit
count=$FRAME_START count=$FRAME_START
for range in "${fileRanges[@]}"; do #Go through the subfolders sequentially for range in "${fileRanges[@]}"; do #Go through the subfolders sequentially
@ -885,6 +1015,7 @@ for ARG in $*; do
done done
rm -r $tmpOut #Remove the now empty subfolder rm -r $tmpOut #Remove the now empty subfolder
done done
#~ exit
elif [ $EXT == "RAW" ] || [ $EXT == "raw" ]; then elif [ $EXT == "RAW" ] || [ $EXT == "raw" ]; then
echo -e $rawStat echo -e $rawStat
@ -904,6 +1035,8 @@ for ARG in $*; do
bad_name="badpixels_${TRUNC_ARG}.txt" bad_name="badpixels_${TRUNC_ARG}.txt"
gen_bad="${TMP}/${bad_name}" gen_bad="${TMP}/${bad_name}"
touch $bad_name
#~ exit
if [ $EXT == "MLV" ] || [ $EXT == "mlv" ]; then if [ $EXT == "MLV" ] || [ $EXT == "mlv" ]; then
$MLV_BP -o $gen_bad $ARG $MLV_BP -o $gen_bad $ARG
@ -931,7 +1064,6 @@ for ARG in $*; do
gen_bad="${TMP}/${bad_name}" gen_bad="${TMP}/${bad_name}"
cp $BADPIXEL_PATH "${gen_bad}" cp $BADPIXEL_PATH "${gen_bad}"
BADPIXELS="-P ${gen_bad}" BADPIXELS="-P ${gen_bad}"
#~ echo $gen_bad
fi fi
#Dual ISO Conversion #Dual ISO Conversion
@ -1000,10 +1132,11 @@ for ARG in $*; do
echo -e "\e[1m${TRUNC_ARG}:\e[0m Generating WB...\n" echo -e "\e[1m${TRUNC_ARG}:\e[0m Generating WB...\n"
#Calculate n, the distance between samples. #Calculate n, the distance between samples.
if [ $WHITE_SPD -gt $FRAMES ]; then frameLen=$(echo "$FRAME_END - $FRAME_START" | bc)
WHITE_SPD=$FRAMES if [ $WHITE_SPD -gt $frameLen ]; then
WHITE_SPD=$frameLen
fi fi
n=`echo "${FRAMES} / ${WHITE_SPD}" | bc` n=`echo "${frameLen} / ${WHITE_SPD}" | bc`
toBal="${TMP}/toBal" toBal="${TMP}/toBal"
mkdirS $toBal mkdirS $toBal
@ -1019,7 +1152,7 @@ for ARG in $*; do
mv "$TMP/${name%.*}.tiff" $toBal #TIFF MOVEMENT. We use TIFFs here because it's easy for dcraw and Python. mv "$TMP/${name%.*}.tiff" $toBal #TIFF MOVEMENT. We use TIFFs here because it's easy for dcraw and Python.
let t++ let t++
fi fi
echo -e "\e[2K\rWB Development: Sample ${t}/$(echo "${FRAMES} / $n" | bc) (Frame: $(echo "${i} + 1" | bc)/${FRAMES})\c" echo -e "\e[2K\rWB Development: Sample ${t}/$(echo "${frameLen} / $n" | bc) (Frame: $(echo "${i} + 1" | bc)/${FRAME_END})\c"
let i++ let i++
done done
echo "" echo ""
@ -1222,10 +1355,9 @@ for ARG in $*; do
fi fi
fi fi
echo -e "\n\e[1mCleaning Up.\e[0m\n"
#Potentially move DNGs. #Potentially move DNGs.
if [ $KEEP_DNGS == true ]; then if [ $KEEP_DNGS == true ]; then
echo -e "\e[1mMoving DNGs...\e[0m"
DNG="${FILE}/dng_${TRUNC_ARG}" DNG="${FILE}/dng_${TRUNC_ARG}"
mkdirS $DNG mkdirS $DNG
@ -1237,6 +1369,8 @@ for ARG in $*; do
fi fi
fi fi
echo -e "\n\e[1mCleaning Up.\e[0m\n\n"
#Delete tmp #Delete tmp
rm -rf $TMP rm -rf $TMP

42
sRange.py 100644
View File

@ -0,0 +1,42 @@
#!/usr/bin/env python3
#~ The MIT License (MIT)
#~ Copyright (c) 2016 Sofus Rose
#~ Permission is hereby granted, free of charge, to any person obtaining a copy
#~ of this software and associated documentation files (the "Software"), to deal
#~ in the Software without restriction, including without limitation the rights
#~ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
#~ copies of the Software, and to permit persons to whom the Software is
#~ furnished to do so, subject to the following conditions:
#~
#~ The above copyright notice and this permission notice shall be included in all
#~ copies or substantial portions of the Software.
#~
#~ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
#~ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
#~ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
#~ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
#~ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
#~ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
#~ SOFTWARE.
import sys
def spSeq(seq, outLen) :
perfSep = (1/outLen) * len(seq)
return list(filter(len, [seq[round(perfSep * i):round(perfSep * (i + 1))] for i in range(len(seq))]))
def splitThreadRange(inNum, inThreads) :
return [str(l[0]) + '-' + str(l[-1]) for l in spSeq(list(range(inNum)), inThreads)]
if __name__ == "__main__" :
num = int(sys.argv[1])
threads = int(sys.argv[2])
print(*splitThreadRange(num, threads))
#mlv_dump cannot use 0-0.