so-rose
/
convmlv
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 3 Wiki Activity

Huge color management release. Added lists getting ready for true compatibility, multiple LUTs, dcraw paths, updated convmlv -h. 

Bug Fixes: Reuse of DNGs, DPX colorspace bug, PNG compression format, "files left" display for DNG folders, ARG bug, filter application sequence, file-specific reset bugs.
Error Checking: Though not complete, it's been drastically upscaled - there should be far fewer "weird errors"; it'll tell you what's wrong.
Speedups: ffmpeg filtering is faster for EXRs. movie creation is much faster if images were developed.
modularize
Sofus Albert Høgsbro Rose 2016-07-09 02:11:59 +02:00
parent 8e0f352855
commit fdfce59bc4
43 changed files with 8132492 additions and 446 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
CHANGELOG
View File

@ -1,4 +1,13 @@
[code] [code]
[b]v2.0.0[/b]: I think convmlv is the stablest it's ever been! Though I'm sure there's plenty more bugs to discover :).
[size=8pt]*Color Management: Real, 3D LUT-based gamma/gamut choices! See -g/-G. There is now minimal quality loss from sensor to image. Adds a dependency: color-core. [/size]
[size=8pt]*Extra Colorspaces: Add extra colorspace options with the optional dependency color-ext - it gives logspace, wide-gamut, etc. options. It's not yet complete - stay tuned![/size]
[size=8pt]*More Dependency Lists: Ubuntu, Fedora, and Homebrew lists have been added to the -K option, allowing easy install with those package managers.[/size]
[size=8pt]*Misc: Specify multiple LUTs in a chain, use a custom DCRAW path, updated + streamlined 'convmlv -h' page.[/size]
[size=8pt]*Speedups: ffmpeg filtering is faster for EXRs. movie creation is much faster if images were developed.[/size]
[size=8pt]*Error Checking: Though not complete, it's been drastically upscaled - there should be far fewer "weird errors"; it'll tell you what's wrong.[/size]
[size=8pt]*Bug Fixes: Reuse of DNGs, DPX colorspace bug, PNG compression format, "files left" display for DNG folders, ARG bug, filter application sequence, file-specific reset bugs.[/size]
[b]v1.9.4[/b]: [url=https://bitbucket.org/so-rose/convmlv/src]https://bitbucket.org/so-rose/convmlv/src[/url] [b]v1.9.4[/b]: [url=https://bitbucket.org/so-rose/convmlv/src]https://bitbucket.org/so-rose/convmlv/src[/url]
[size=8pt]*Added Unsharp filter. Useful for counteracting all 6 denoisers ;).[/size] [size=8pt]*Added Unsharp filter. Useful for counteracting all 6 denoisers ;).[/size]
[size=8pt]*Bug Fixes/Small Improvements: File-specific blocks/multiple files fixed. Filter application has been sanitized.[/size] [size=8pt]*Bug Fixes/Small Improvements: File-specific blocks/multiple files fixed. Filter application has been sanitized.[/size]

184
README.md
View File

@ -6,30 +6,40 @@ See [http://www.magiclantern.fm/forum/index.php?topic=16799.0](http://www.magicl
**The pdf found in the source, under docs->docs.pdf, is very outdated. Use the help text below.** **The pdf found in the source, under docs->docs.pdf, is very outdated. Use the help text below.**
Help page is below: Full help page is below:
```text ```text
Usage: Usage:
./convmlv.sh [FLAGS] [OPTIONS] files ./convmlv.sh [FLAGS] [OPTIONS] files
INFO: INFO:
A script allowing you to develop ML files into workable formats. Many useful options are exposed. A program allowing you to develop ML files into workable formats. Many useful options are exposed.
--> Image Defaults: Compressed 16-bit Linear EXR. --> Defaults: Compressed 16-bit Linear EXR. 10-bit Prores4444 MOV.
--> Acceptable Inputs: MLV, RAW, DNG Folder. --> Color Defaults: Linear (1.0) Gamma on sRGB Gamut, using Camera White Balance.
--> Option Input: From command line or config file.
--> Acceptable Inputs: MLV, RAW (requires raw2dng), Folder containing DNGs.
--> Option Input: From command line or config file (specify with -C).
--> Forum Post: http://www.magiclantern.fm/forum/index.php?topic=16799. --> Forum Post: http://www.magiclantern.fm/forum/index.php?topic=16799.
--> A note: BE CAREFUL WITH OPTIONS. Wrong values will give very strange errors. Read this page well!!
It's as simple or complex as you need it to be: 'convmlv -m <mlvfile>.mlv' is enough for good-looking output!
VERSION: 1.9.4 VERSION: 2.0.0
MANUAL DEPENDENCIES: MANUAL DEPENDENCIES:
-mlv_dump: Required. http://www.magiclantern.fm/forum/index.php?topic=7122.0 Place these in RES_PATH (see OPTIONS, BASIC). Keep in mind you also need dist. and pip packages.
-raw2dng: For DNG extraction from RAW. http://www.magiclantern.fm/forum/index.php?topic=5404.0 --> See 'Dist Deps' and 'Python Deps'
-mlv2badpixels.sh: For bad pixel removal. https://bitbucket.org/daniel_fort/ml-focus-pixels/src
-cr2hdr: For Dual ISO Development. Two links: http://www.magiclantern.fm/forum/index.php?topic=16799.0 -- mlv_dump: Required binary. http://www.magiclantern.fm/forum/index.php?topic=7122.0
-sRange.py: Required. See convmlv repository. -- color-core: Required folder of LUTs. See convmlv repository.
-balance.py: For Auto White Balance. See convmlv repository. -- sRange.py: Required script. See convmlv repository.
-- raw2dng: For DNG extraction from RAW. http://www.magiclantern.fm/forum/index.php?topic=5404.0
-- mlv2badpixels.sh: For bad pixel removal. https://bitbucket.org/daniel_fort/ml-focus-pixels/src
-- cr2hdr: For Dual ISO Development. Two links: http://www.magiclantern.fm/forum/index.php?topic=16799.0
-- balance.py: For Auto White Balance. See convmlv repository.
-- color-ext: Extra LUTs, providing more color resources. See convmlv repository.
OPTIONS, BASIC: OPTIONS, BASIC:
-v, --version version - Print out version string. -v, --version version - Print out version string.
@ -39,60 +49,65 @@ OPTIONS, BASIC:
-o, --outdir <path> OUTDIR - The path in which files will be placed. -o, --outdir <path> OUTDIR - The path in which files will be placed.
-P, --res-path <path> RES_PATH - The path in which all manual dependencies are looked for. -P, --res-path <path> RES_PATH - The path in which all manual dependencies are looked for.
--> Default: Current Directory.
--dcraw <path> DCRAW - The path to dcraw.
--mlv-dump <path> MLV_DUMP - The path to mlv_dump. --mlv-dump <path> MLV_DUMP - The path to mlv_dump.
--raw-dump <path> RAW_DUMP - The path to raw2dng. --raw-dump <path> RAW_DUMP - The path to raw2dng.
--badpixels <path> MLV_BP - The path to mlv2badpixels.sh (by dfort). --badpixels <path> MLV_BP - The path to mlv2badpixels.sh (by dfort).
--cr-hdr <path> CR_HDR - The path to cr2hdr. --cr-hdr <path> CR_HDR - The path to cr2hdr.
--srange <path> SRANGE - The path to sRange.py. --srange <path> SRANGE - The path to sRange.py.
--balance <path> BAL - The path to balance.py. --balance <path> BAL - The path to balance.py.
--python <path> PYTHON - The path or command used to invoke Python. --python <path> PYTHON - The path or command used to invoke Python 3. Default is python3 on Linux, python on Mac.
-T, --threads [int] THREADS - Override amount of utilized process threads -T, --threads [int] THREADS - Override amount of utilized process threads. Default is MAX - 1.
OPTIONS, OUTPUT: OPTIONS, OUTPUT:
-i IMAGE - Will output image sequence. -i IMAGE - Will output image sequence.
-t [0:3] IMG_FMT - Specified image output format. -t [0:3] IMG_FMT - Image output format.
--> 0: EXR (default), 1: TIFF, 2: PNG, 3: Cineon (DPX)." --> 0: EXR (default), 1: TIFF, 2: PNG, 3: Cineon (DPX)."
--> Note: Only EXR supports Linear output. Specify -g 3 if not using EXR.
-m MOVIE - Will output a Prores4444 file. -m MOVIE - Will output a Prores4444 file.
-p [0:3] PROXY - Create proxies alongside main output. -p [0:3] PROXY - Create proxies alongside main output.
--> 0: No proxies (Default). 1: H.264 proxy. 2: JPG proxy sequence. 3: Both. --> 0: No proxies (Default). 1: H.264 proxy. 2: JPG proxy sequence. 3: Both.
--> JPG proxy *won't* be developed w/o IMAGE. H.264 proxy *will* be developed no matter what, if specified.
--> JPG proxy will always be in sRGB Gamma/sRGB Gamut. H.264 proxy is color managed.
--> JPG proxy *won't* be developed w/o IMAGE. H.264 proxy *will* be developed no matter what, if specified here.
--> Why? JPG is for potential use in editing. H.264 is for a quick visual preview of convmlv's output.
-s [0%:100%] PROXY_SCALE - the size, in %, of the proxy output. -s [0%:100%] PROXY_SCALE - the size, in %, of the proxy output.
--> 50% is default. --> Default: 50%.
-k KEEP_DNGS - Specify if you want to keep the DNG files. -k KEEP_DNGS - Specify if you want to keep the DNG files.
--> Run convmlv on the top level folder of former output to reuse saved DNGs from that run! --> Run convmlv on the top level folder of former output to reuse saved DNGs from that run!
-r <start>-<end> FRAME_RANGE - Specify to output this frame range only. -r <start>-<end> FRAME_RANGE - Specify to process an integer frame range.
--> You may use s and e, such that s = start frame, e = end frame. --> You may use the characters 's' and 'e', such that s = start frame, e = end frame.
--> Indexed from 0 to (# of frames - 1). --> Indexed from 0 to (# of frames - 1). Develops from 1 to ($ of frames)
--> A single number may be writted to develop that frame only. --> A single number may be writted to develop that single frame.
--> DO NOT try to reuse DNGs while developing a larger frame range.
--uncompress UNCOMP - Turns off lossless image compression. Otherwise: --uncompress UNCOMP - Turns off lossless image compression. Otherwise:
--> TIFF: ZIP, EXR: PIZ, PNG: lvl 9 (zlib deflate), DPX: RLE. --> TIFF: ZIP, EXR: PIZ, PNG: lvl 0, DPX: RLE.
OPTIONS, RAW DEVELOPMENT: OPTIONS, RAW DEVELOPMENT:
-d [0:3] DEMO_MODE - Demosaicing algorithm. Higher modes are slower + better. -d [0:3] DEMO_MODE - Demosaicing algorithm. Higher modes are slower + better.
--> 0: Bilinear. 1: VNG (default). 2: PPG. 3: AHD. --> 0: Bilinear. 1: VNG (default). 2: PPG. 3: AHD.
-f FOUR_COLOR - Interpolate as RGBG. Can often fix weirdness with VNG/AHD. -f FOUR_COLOR - Interpolate as RGBG. Fixes weirdness with VNG/AHD, at the cost of sharpness.
-H [0:9] HIGHLIGHT_MODE - Highlight management options. -H [0:9] HIGHLIGHT_MODE - Highlight management options.
--> 0: White, clipped highlights. 1: Clipped, colored highlights. 2: Similar to 1, but adjusted to grey. --> 0: White, clipped highlights. 1: Unclipped but colored highlights. 2: The defail of 1, but adjusted to grey.
--> 3-9: Highlight reconstruction. Can cause flickering; 1 or 2 usually give better results. --> 3-9: Highlight reconstruction. Can cause flickering. Start at 5, then adjust to color (down) or to white (up).
-c [0:3] CHROMA_SMOOTH - Apply shadow/highlight chroma smoothing to the footage. -c [0:3] CHROMA_SMOOTH - Apply shadow/highlight chroma smoothing to the footage.
--> 0: None (default). 1: 2x2. 2: 3x3. 3: 5x5. --> 0: None (default). 1: 2x2. 2: 3x3. 3: 5x5.
--> MLV Only. --> MLV input Only.
-n [int] WAVE_NOISE - Apply wavelet denoising. -n [int] WAVE_NOISE - Apply wavelet denoising.
--> Default: None. Subtle: 25. Medium: 50. Strong: 125. --> Default: None. Subtle: 25. Medium: 50. Strong: 125.
@ -108,76 +123,130 @@ OPTIONS, RAW DEVELOPMENT:
--> Option Value: <LS>-<CS>:<LT>-<CT> --> Option Value: <LS>-<CS>:<LT>-<CT>
--> Weak: 2-1:2-3. Medium: 3-2:2-3. Strong: 7-7:5-5 --> Weak: 2-1:2-3. Medium: 3-2:2-3. Strong: 7-7:5-5
--> DONT combine with TEMP_NOISE.
-G [i-i-i-i] REM_NOISE - Yet another spatial denoiser, with 4 choices of 24 modes. -O [i-i-i-i] REM_NOISE - Yet another spatial denoiser, with 4 choices of 24 modes.
--> See https://ffmpeg.org/ffmpeg-filters.html#removegrain for list of modes. --> See https://ffmpeg.org/ffmpeg-filters.html#removegrain for list of modes.
--> Option Value: <mode1>-<mode2>-<mode3>-<mode4> --> Option Value: <mode1>-<mode2>-<mode3>-<mode4>
--> I truly cannot tell you what values will be helpful to you; there are too many... Look at the link above! --> I truly cannot tell you what values will be helpful to you; there are too many... Look at the link!
-g [0:4] SPACE - Output color transformation. --shallow SHALLOW - Output smaller, 8-bit files.
--> 0: Linear. 1: 2.2 (Adobe RGB). 2: 1.8 (ProPhoto RGB). 3: sRGB. 4: BT.709. --> Read why this is a bad idea: http://www.cambridgeincolour.com/tutorials/bit-depth.htm
--shallow SHALLOW - Output 8-bit files.
OPTIONS, COLOR: OPTIONS, COLOR:
-g [0:4] GAMMA - Output gamma. A curve applied to the output, for easier viewing/grading.
--> 0: Standard (Around 2.2). 1: Linear (Default).
--> Requires color-ext: 2: Cineon. 3: C-Log2 4: S-Log3
--> "Standard" grades to the gamut specification, and to 2.2 if that's not given.
-G [0:6] GAMUT - Output gamut. The range of colors that can exist in the output.
--> 0: sRGB (Default). 1: Adobe RGB. 2: Rec.709. 3: XYZ (Always Linear Gamma).
--> Requires color-ext: 4: Rec2020 5: DCI-P3 6: Sony S-Gamut3.cine
-w [0:2] WHITE - This is a modal white balance setting. -w [0:2] WHITE - This is a modal white balance setting.
--> 0: Auto WB. 1: Camera WB (default). 2: No Change. --> 0: Auto WB (requires balance.py). 1: Camera WB (default). 2: No Change.
--> AWB uses the Grey's World algorithm.
-A [i:i:i:i] SHARP - Lets you sharpen, or blur, your footage. -A [i:i:i:i] SHARP - Lets you sharpen, or blur, your footage.
--> BE CAREFUL. Wrong values will give you strange errors.
--> Size/Strength (S/T). S is the size of the sharpen/blur effect, T is the strength of the sharpen/blur effect. --> Size/Strength (S/T). S is the size of the sharpen/blur effect, T is the strength of the sharpen/blur effect.
--> Luma/Chroma (L/C). L is the detail, C is the color. Luma sharpening more effective. --> Luma/Chroma (L/C). L is the detail, C is the color. Luma sharpening more effective.
--> Tip: Chroma blur can actually be a helpful noise reduction technique.
--> Option Value: <LS>:<LT>:<CS>:<CT> --> Option Value: <LS>:<LT>:<CS>:<CT>
--> LS and CS must be ODD, between 3 and 63. Negative LT/CT values blur, while positive ones sharpen. --> LS and CS must be ODD, between 3 and 63. Negative LT/CT values blur, while positive ones sharpen.
--> Strong Sharp: 7:3:7:3 Strong Blur: 7,-3:7,-3 --> Strong Sharp: 7:3:7:3 Strong Blur: 7,-3:7,-3. Medium Sharp: 5:1:3:0
-l <path> LUT - Specify a LUT to apply. -l <path> LUT - Specify a LUT to apply after Color Management.
--> Supports cube, 3dl, dat, m3d. --> Supports cube, 3dl, dat, m3d.
--> Specify -l multiple times, to apply multiple LUTs in sequence.
-S [int] SATPOINT - Specify the 14-bit saturation point of your camera. You don't usually want to. -S [int] SATPOINT - Specify the 14-bit uint saturation point of your camera. You don't usually need to.
--> Lower if -H1 yields purple highlights. Must be correct for highlight reconstruction. --> Worth setting globally, as it's a per-camera setting. Must be correct for highlight reconstruction/unclipped highlights.
--> Determine using the max value of 'dcraw -D -j -4 -T' --> Lower from 15000 if -H1 yields purple highlights, until they turn white.
--> You can determine the optimal value using the max pixel value of 'dcraw -D -j -4 -T'.
--white-speed [int] WHITE_SPD - Samples used to calculate AWB --white-speed [int] WHITE_SPD - Manually specify samples used to calculate AWB.
--allow-white-clip WHITE_CLIP - Let White Balance multipliers clip. --allow-white-clip WHITE_CLIP - Let the White Balance multipliers clip.
OPTIONS, FEATURES: OPTIONS, FEATURES:
-D DESHAKE - Stabilize the video using the wonderful ffmpeg "deshake" module. -D DESHAKE - Auto-stabilize the video using ffmpeg's "deshake" module.
--> You'll probably wish to crop/scale the output in editing, to avoid edge artifacts. --> You may wish to crop/scale the output later, to avoid edge artifacts.
-u DUAL_ISO - Process as dual ISO. -u DUAL_ISO - Process as dual ISO.
--> Requires cr2hdr.
-b BADPIXELS - Fix focus pixels issue using dfort's script. -b BADPIXELS - Fix focus pixels issue using dfort's script.
--> Requires mlv2badpixels.sh.
-a <path> BADPIXEL_PATH - Use your own .badpixels file. -a <path> BADPIXEL_PATH - Use your own .badpixels file. Does NOT require mlv2badpixels.sh
--> How to: http://www.dl-c.com/board/viewtopic.php?f=4&t=686 --> How to: http://www.dl-c.com/board/viewtopic.php?f=4&t=686
-F <path> DARKFRAME - This is the path to a "dark frame MLV"; effective for noise reduction. -F <path> DARKFRAME - This is the path to a "dark frame MLV"; effective for noise reduction.
--> How to: Record 5 sec w/lens cap on & same settings as footage. Pass MLV in here. --> How to: Record 5 sec w/lens cap on & same settings as footage. Pass MLV in here.
--> If the file extension is '.darkframe', the file will be used as the preaveraged dark frame. --> If the file extension is '.darkframe', the file will be used as a preaveraged dark frame.
-R <path> dark_out - Specify to create a .darkframe file from passed in MLV. -R <path> darkframe_output - Specify to create a .darkframe file from passed in MLV.
--> Usage: 'convmlv -R <path> <input>.MLV' --> Usage: 'convmlv -R <path> <input>.MLV'
--> Averages <input>.MLV to create <path>.darkframe. --> Averages <input>.MLV to create <path>.darkframe.
--> THE .darkframe EXTENSION IS ADDED FOR YOU. --> THE .darkframe EXTENSION IS ADDED FOR YOU.
OPTIONS, INFO: OPTIONS, INFO:
-q Output MLV settings. -q settings - Output MLV settings.
-K Debian Package Deps - Output package dependecies. -K [0:3] Dist Deps - Output package dependecies, for use with common package managers.
--> Install (Debian only): sudo apt-get install $ (./convmlv -K) --> 0: Debian, 1: Ubuntu, 2: Fedora, 3: Homebrew (Mac)
--> Deps Install (Debian): sudo apt-get install $(./convmlv.sh -K 0)
--> Deps Install (Ubuntu): sudo apt-get install $(./convmlv.sh -K 1)
--> Deps Install (Fedora): sudo yum install $(./convmlv.sh -K 2)
--> Deps Install (Homebrew Mac): brew install $(./convmlv.sh -K 3)
-Y Python Deps - Lists Python dependencies. Works directly with pip. -Y Python Deps - Lists Python dependencies. Works directly with pip.
-->Install (Linux): sudo pip3 install $ (./convmlv -Y) -->Install (Cross-Platform): sudo python3 -m pip install $ (./convmlv -Y)
-M Manual Deps - Lists manual dependencies, which must be downloaded by hand. -M Manual Deps - Lists manual dependencies, which must be downloaded by hand.
--> There's no automatic way to install these. See http://www.magiclantern.fm/forum/index.php?topic=16799.0 . --> Manually place all in RES_PATH. See http://www.magiclantern.fm/forum/index.php?topic=16799.0 .
COLOR MANAGEMENT:
INTRO Images aren't simple. They are often stored, processed, and viewed as a result of complex transformations usually called,
in applications, color management. Understanding it is required as a colourist, and encouraged as a DPs and Cinematographers.
--> Intro: http://www.cambridgeincolour.com/tutorials/color-management1.htm
--> Understanding Gamma: http://www.cambridgeincolour.com/tutorials/gamma-correction.htm
--> Color Spaces: http://www.cambridgeincolour.com/tutorials/color-spaces.htm
--> Conversions: http://www.cambridgeincolour.com/tutorials/color-space-conversion.htm
--> Monitor Calibration: http://www.cambridgeincolour.com/tutorials/monitor-calibration.htm
PIPELINE convmlv is a color managed application, designed to retain quality from RAW footage:
-- mlv_dump writes camera-specific color transformation matrices as metadata in developed DNGs.
--> This defines the camera's gamut.
-- dcraw applies these matrices, then transforms the newly developed image to the super-wide XYZ colorspace.
--> All color detail is preserved, and if now in a well-defined colorspace.
--> No gamma has been applied - the image is now Linear XYZ.
-- ffmpeg applies the specified (up to) x64 resolution 3D LUTs, in .cube format.
--> Use -g and -G to specify Gamma/Gamut combinations.
--> The output employs DATA, not LEGAL, values. The resulting image is flatter, but retains all shadow/highlight detail.
--> -l specified LUTs are applied afterwards.
3D LUTS The included LUTs, found in the convmlv repository, are key to convmlv's color management solution:
-- color-core: The required LUTs, including sRGB (default), Adobe RGB, and Rec709 in Standard and Linear gamma.
-- color-ext: Optional LUTs, including Rec2020, DCI-P3, etc. in Standard/Linear gammas, but also in Log formats.
Create Your Own LUTs using LUTCalc, for any grading format output: https://cameramanben.github.io/LUTCalc/ (watch his tutorials).
--> Note that convmlv only accepts up to 64x64x64 LUTs. You can resize LUTs using pylut (https://pypi.python.org/pypi/pylut).
--> The pylut command to resize is 'pylut <yourx65lut>.cube --resize 64'. Alternatively, you can use pylut from Python (2X only).
--> I reccommend Legal --> Data LUTs, as this conserves shadow/highlight detail for grading. Legal --> Legal looks better, but with detail loss.
CONFIG FILE: CONFIG FILE:
Config files, another way to specify options, can save you time & lend you convenience in production situations. Config files, another way to specify options, can save you time & lend you convenience in production situations.
@ -187,7 +256,7 @@ CONFIG FILE:
SYNTAX: SYNTAX:
Most options listed above have an uppercased VARNAME, ex. OUTDIR. Yu can specify such options in config files, as such: Most options listed above have an uppercased VARNAME, ex. OUTDIR. You can specify such options in config files, as such:
<VARNAME> <VALUE> <VARNAME> <VALUE>
@ -199,7 +268,7 @@ CONFIG FILE:
CONFIG_NAME <name> CONFIG_NAME <name>
Flags If the value is a true/false flag (ex. IMAGE), simply specifying VARNAME is enough. THere is no VALUE. Flags If the value is a true/false flag (ex. IMAGE), simply specifying VARNAME is enough. There is no VALUE.
OPTION ORDER OF PRECEDENCE Options override each other as such: OPTION ORDER OF PRECEDENCE Options override each other as such:
-LOCAL options overwrite GLOBAL options. -LOCAL options overwrite GLOBAL options.
@ -220,6 +289,9 @@ CONFIG FILE:
Contact me with any feedback or questions at convmlv@sofusrose.com, or PM me (so-rose) on the ML forums! Contact me with any feedback or questions at convmlv@sofusrose.com, PM me (so-rose) on the ML forums, or post on the thread!
``` ```
Run 'sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g" docs/MANPAGE' to strip colors from MANPAGE after generating through less.

6
balance.py
View File

@ -28,7 +28,7 @@ import sys, os
from glob import glob from glob import glob
import numpy as np import numpy as np
from PIL import Image #~ from PIL import Image
import tifffile as tff import tifffile as tff
def grey(numImg) : def grey(numImg) :
@ -43,8 +43,8 @@ def grey(numImg) :
def imgOpen(path) : def imgOpen(path) :
if path[-4:] == '.tif' or path[-4:] == 'tiff' : if path[-4:] == '.tif' or path[-4:] == 'tiff' :
return tff.TiffFile(path).asarray() return tff.TiffFile(path).asarray()
else : #~ else :
return np.asarray(Image.open(path).convert('RGB')) #~ return np.asarray(Image.open(path).convert('RGB'))
def greyAvg(paths) : def greyAvg(paths) :
wb = np.asarray([grey(imgOpen(path)) for path in paths]) wb = np.asarray([grey(imgOpen(path)) for path in paths])

262145
color-core/lin_xyz--lin_argb.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-core/lin_xyz--lin_rec709.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-core/lin_xyz--lin_srgb.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-core/lin_xyz--lin_xyz.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-core/lin_xyz--rec709_rec709.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-core/lin_xyz--srgb_srgb.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-core/lin_xyz--y2*2_argb.cube 100644
View File

File diff suppressed because it is too large Load Diff

4916
color-core/srgb--lin.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--cineon_argb.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--cineon_dcip3.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--cineon_rec2020.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--cineon_rec709.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--cineon_srgb.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--cineon_ssg3c.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--clog2_argb.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--clog2_dcip3.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--clog2_rec2020.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--clog2_rec709.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--clog2_srgb.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--clog2_ssg3c.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--dcip3_dcip3.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--lin_dcip3.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--lin_rec2020.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--lin_ssg3c.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--rec2020_rec2020.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--slog3_argb.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--slog3_dcip3.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--slog3_rec2020.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--slog3_rec709.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--slog3_srgb.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--slog3_ssg3c.cube 100644
View File

File diff suppressed because it is too large Load Diff

262145
color-ext/lin_xyz--y2*2_ssg3c.cube 100644
View File

File diff suppressed because it is too large Load Diff

26
configs/atadenoise-test.conf
View File

@ -17,16 +17,17 @@ PYTHON python3
### <------------------ OUTPUT OPTIONS ------------------> ### <------------------ OUTPUT OPTIONS ------------------>
IMAGE IMAGE
IMG_FMT 0 IMG_FMT 3
#~ MOVIE MOVIE
#~ PROXY 3 PROXY 3
PROXY_SCALE 50% PROXY_SCALE 100%
KEEP_DNGS KEEP_DNGS
#~ UNCOMP #~ UNCOMP
FRAME_RANGE 50-55
### <------------------ RAW OPTIONS ------------------> ### <------------------ RAW OPTIONS ------------------>
DEMO_MODE 3 DEMO_MODE 3
@ -34,26 +35,29 @@ DEMO_MODE 3
#~ CHROMA_SMOOTH 1 #~ CHROMA_SMOOTH 1
HIGHLIGHT_MODE 7 HIGHLIGHT_MODE 7
#~ TEMP_NOISE 0.1-0.04 #~ TEMP_NOISE 0.1-0.04
HQ_NOISE 3-2:3-4 #~ HQ_NOISE 3-2:3-4
REM_NOISE 1-1-1-1 #~ REM_NOISE 1-1-1-1
SPACE 0 GAMMA 0
GAMUT 0
### <------------------ COLOR OPTIONS ------------------> ### <------------------ COLOR OPTIONS ------------------>
WHITE 1 WHITE 1
SHARP 7:5:5:3 #~ SHARP 7:5:5:3
### <------------------ FEATURE OPTIONS ------------------> ### <------------------ FEATURE OPTIONS ------------------>
#~ DESHAKE #~ DESHAKE
#~ DUAL_ISO
DARKFRAME /home/sofus/subhome/src/convmlv/darkframes_7D/f2.8_iso2500_crop1080.darkframe DARKFRAME /home/sofus/subhome/src/convmlv/darkframes_7D/f2.8_iso2500_crop1080.darkframe
BADPIXEL_PATH ./7D_badpixels.txt BADPIXEL_PATH ./7D_badpixels.txt
#~ BADPIXELS
/ test_girl_noise / test_girl_noise
LUT /home/sofus/subhome/src/convmlv/test_footage/girl_lut_test.cube #~ LUT /home/sofus/subhome/src/convmlv/color-core/gamut_gamma.cube
FRAME_RANGE 456 #~ LUT /home/sofus/subhome/src/convmlv/color-ext/lin_xyz--clog2_srgb.cube
FRAME_RANGE 348-355
* *
/ M03-1714 / M03-1714

3
configs/dng_dump.conf
View File

@ -24,7 +24,8 @@ HIGHLIGHT_MODE 0
CHROMA_SMOOTH 1 CHROMA_SMOOTH 1
WAVE_NOISE 50 WAVE_NOISE 50
SPACE 0 GAMMA 1
GAMUT 6
#SHALLOW #SHALLOW
### <------------------ COLOR OPTIONS ------------------> ### <------------------ COLOR OPTIONS ------------------>

18
configs/global.conf
View File

@ -8,13 +8,13 @@ CONFIG_NAME global_hq
OUTDIR ./raw_conv OUTDIR ./raw_conv
RES_PATH . RES_PATH .
MLV_DUMP ./mlv_dump #~ MLV_DUMP ./mlv_dump
RAW_DUMP ./raw2dng #~ RAW_DUMP ./raw2dng
MLV_BP ./mlv2badpixels.sh #~ MLV_BP ./mlv2badpixels.sh
CR_HDR ./cr2hdr #~ CR_HDR ./cr2hdr
SRANGE ./sRange.py #~ SRANGE ./sRange.py
BAL ./balance.py #~ BAL ./balance.py
PYTHON python3 #~ PYTHON python3
### <------------------ OUTPUT OPTIONS ------------------> ### <------------------ OUTPUT OPTIONS ------------------>
@ -26,7 +26,7 @@ IMG_FMT 0
#MOVIE #MOVIE
PROXY 0 PROXY 0
PROXY_SCALE 50% PROXY_SCALE 75%
### <------------------ RAW OPTIONS ------------------> ### <------------------ RAW OPTIONS ------------------>
@ -37,4 +37,4 @@ DEMO_MODE 3
WHITE 1 WHITE 1
### <------------------ FEATURE OPTIONS ------------------> ### <------------------ FEATURE OPTIONS ------------------>
#DARKFRAME ./df.darkframe #~ DARKFRAME ./df.darkframe

5
configs/template.txt
View File

@ -43,12 +43,13 @@ TEMP_NOISE 0.03-0.04
HQ_NOISE 2-1:2-3 HQ_NOISE 2-1:2-3
#REM_NOISE 1-1-1-1 #REM_NOISE 1-1-1-1
SPACE 3 GAMMA 1
GAMUT 6
#SHALLOW #SHALLOW
### <------------------ COLOR OPTIONS ------------------> ### <------------------ COLOR OPTIONS ------------------>
WHITE 0 WHITE 1
SHARP 7:3:0:0 SHARP 7:3:0:0
#LUT lut.cube #LUT lut.cube

1003
convmlv.sh
View File

File diff suppressed because it is too large Load Diff

178
docs/MANPAGE
View File

@ -2,22 +2,33 @@ Usage:
./convmlv.sh [FLAGS] [OPTIONS] files ./convmlv.sh [FLAGS] [OPTIONS] files
INFO: INFO:
A script allowing you to develop ML files into workable formats. Many useful options are exposed. A program allowing you to develop ML files into workable formats. Many useful options are exposed.
--> Image Defaults: Compressed 16-bit Linear EXR. --> Defaults: Compressed 16-bit Linear EXR. 10-bit Prores4444 MOV.
--> Acceptable Inputs: MLV, RAW, DNG Folder. --> Color Defaults: Linear (1.0) Gamma on sRGB Gamut, using Camera White Balance.
--> Option Input: From command line or config file.
--> Acceptable Inputs: MLV, RAW (requires raw2dng), Folder containing DNGs.
--> Option Input: From command line or config file (specify with -C).
--> Forum Post: http://www.magiclantern.fm/forum/index.php?topic=16799. --> Forum Post: http://www.magiclantern.fm/forum/index.php?topic=16799.
--> A note: BE CAREFUL WITH OPTIONS. Wrong values will give very strange errors. Read this page well!!
It's as simple or complex as you need it to be: 'convmlv -m <mlvfile>.mlv' is enough for good-looking output!
VERSION: 1.9.4 VERSION: 2.0.0
MANUAL DEPENDENCIES: MANUAL DEPENDENCIES:
-mlv_dump: Required. http://www.magiclantern.fm/forum/index.php?topic=7122.0 Place these in RES_PATH (see OPTIONS, BASIC). Keep in mind you also need dist. and pip packages.
-raw2dng: For DNG extraction from RAW. http://www.magiclantern.fm/forum/index.php?topic=5404.0 --> See 'Dist Deps' and 'Python Deps'
-mlv2badpixels.sh: For bad pixel removal. https://bitbucket.org/daniel_fort/ml-focus-pixels/src
-cr2hdr: For Dual ISO Development. Two links: http://www.magiclantern.fm/forum/index.php?topic=16799.0 -- mlv_dump: Required binary. http://www.magiclantern.fm/forum/index.php?topic=7122.0
-sRange.py: Required. See convmlv repository. -- color-core: Required folder of LUTs. See convmlv repository.
-balance.py: For Auto White Balance. See convmlv repository. -- sRange.py: Required script. See convmlv repository.
-- raw2dng: For DNG extraction from RAW. http://www.magiclantern.fm/forum/index.php?topic=5404.0
-- mlv2badpixels.sh: For bad pixel removal. https://bitbucket.org/daniel_fort/ml-focus-pixels/src
-- cr2hdr: For Dual ISO Development. Two links: http://www.magiclantern.fm/forum/index.php?topic=16799.0
-- balance.py: For Auto White Balance. See convmlv repository.
-- color-ext: Extra LUTs, providing more color resources. See convmlv repository.
OPTIONS, BASIC: OPTIONS, BASIC:
-v, --version version - Print out version string. -v, --version version - Print out version string.
@ -27,60 +38,65 @@ OPTIONS, BASIC:
-o, --outdir <path> OUTDIR - The path in which files will be placed. -o, --outdir <path> OUTDIR - The path in which files will be placed.
-P, --res-path <path> RES_PATH - The path in which all manual dependencies are looked for. -P, --res-path <path> RES_PATH - The path in which all manual dependencies are looked for.
--> Default: Current Directory.
--dcraw <path> DCRAW - The path to dcraw.
--mlv-dump <path> MLV_DUMP - The path to mlv_dump. --mlv-dump <path> MLV_DUMP - The path to mlv_dump.
--raw-dump <path> RAW_DUMP - The path to raw2dng. --raw-dump <path> RAW_DUMP - The path to raw2dng.
--badpixels <path> MLV_BP - The path to mlv2badpixels.sh (by dfort). --badpixels <path> MLV_BP - The path to mlv2badpixels.sh (by dfort).
--cr-hdr <path> CR_HDR - The path to cr2hdr. --cr-hdr <path> CR_HDR - The path to cr2hdr.
--srange <path> SRANGE - The path to sRange.py. --srange <path> SRANGE - The path to sRange.py.
--balance <path> BAL - The path to balance.py. --balance <path> BAL - The path to balance.py.
--python <path> PYTHON - The path or command used to invoke Python. --python <path> PYTHON - The path or command used to invoke Python 3. Default is python3 on Linux, python on Mac.
-T, --threads [int] THREADS - Override amount of utilized process threads -T, --threads [int] THREADS - Override amount of utilized process threads. Default is MAX - 1.
OPTIONS, OUTPUT: OPTIONS, OUTPUT:
-i IMAGE - Will output image sequence. -i IMAGE - Will output image sequence.
-t [0:3] IMG_FMT - Specified image output format. -t [0:3] IMG_FMT - Image output format.
--> 0: EXR (default), 1: TIFF, 2: PNG, 3: Cineon (DPX)." --> 0: EXR (default), 1: TIFF, 2: PNG, 3: Cineon (DPX)."
--> Note: Only EXR supports Linear output. Specify -g 3 if not using EXR.
-m MOVIE - Will output a Prores4444 file. -m MOVIE - Will output a Prores4444 file.
-p [0:3] PROXY - Create proxies alongside main output. -p [0:3] PROXY - Create proxies alongside main output.
--> 0: No proxies (Default). 1: H.264 proxy. 2: JPG proxy sequence. 3: Both. --> 0: No proxies (Default). 1: H.264 proxy. 2: JPG proxy sequence. 3: Both.
--> JPG proxy *won't* be developed w/o IMAGE. H.264 proxy *will* be developed no matter what, if specified.
--> JPG proxy will always be in sRGB Gamma/sRGB Gamut. H.264 proxy is color managed.
--> JPG proxy *won't* be developed w/o IMAGE. H.264 proxy *will* be developed no matter what, if specified here.
--> Why? JPG is for potential use in editing. H.264 is for a quick visual preview of convmlv's output.
-s [0%:100%] PROXY_SCALE - the size, in %, of the proxy output. -s [0%:100%] PROXY_SCALE - the size, in %, of the proxy output.
--> 50% is default. --> Default: 50%.
-k KEEP_DNGS - Specify if you want to keep the DNG files. -k KEEP_DNGS - Specify if you want to keep the DNG files.
--> Run convmlv on the top level folder of former output to reuse saved DNGs from that run! --> Run convmlv on the top level folder of former output to reuse saved DNGs from that run!
-r <start>-<end> FRAME_RANGE - Specify to output this frame range only. -r <start>-<end> FRAME_RANGE - Specify to process an integer frame range.
--> You may use s and e, such that s = start frame, e = end frame. --> You may use the characters 's' and 'e', such that s = start frame, e = end frame.
--> Indexed from 0 to (# of frames - 1). --> Indexed from 0 to (# of frames - 1). Develops from 1 to ($ of frames)
--> A single number may be writted to develop that frame only. --> A single number may be writted to develop that single frame.
--> DO NOT try to reuse DNGs while developing a larger frame range.
--uncompress UNCOMP - Turns off lossless image compression. Otherwise: --uncompress UNCOMP - Turns off lossless image compression. Otherwise:
--> TIFF: ZIP, EXR: PIZ, PNG: lvl 9 (zlib deflate), DPX: RLE. --> TIFF: ZIP, EXR: PIZ, PNG: lvl 0, DPX: RLE.
OPTIONS, RAW DEVELOPMENT: OPTIONS, RAW DEVELOPMENT:
-d [0:3] DEMO_MODE - Demosaicing algorithm. Higher modes are slower + better. -d [0:3] DEMO_MODE - Demosaicing algorithm. Higher modes are slower + better.
--> 0: Bilinear. 1: VNG (default). 2: PPG. 3: AHD. --> 0: Bilinear. 1: VNG (default). 2: PPG. 3: AHD.
-f FOUR_COLOR - Interpolate as RGBG. Can often fix weirdness with VNG/AHD. -f FOUR_COLOR - Interpolate as RGBG. Fixes weirdness with VNG/AHD, at the cost of sharpness.
-H [0:9] HIGHLIGHT_MODE - Highlight management options. -H [0:9] HIGHLIGHT_MODE - Highlight management options.
--> 0: White, clipped highlights. 1: Clipped, colored highlights. 2: Similar to 1, but adjusted to grey. --> 0: White, clipped highlights. 1: Unclipped but colored highlights. 2: The defail of 1, but adjusted to grey.
--> 3-9: Highlight reconstruction. Can cause flickering; 1 or 2 usually give better results. --> 3-9: Highlight reconstruction. Can cause flickering. Start at 5, then adjust to color (down) or to white (up).
-c [0:3] CHROMA_SMOOTH - Apply shadow/highlight chroma smoothing to the footage. -c [0:3] CHROMA_SMOOTH - Apply shadow/highlight chroma smoothing to the footage.
--> 0: None (default). 1: 2x2. 2: 3x3. 3: 5x5. --> 0: None (default). 1: 2x2. 2: 3x3. 3: 5x5.
--> MLV Only. --> MLV input Only.
-n [int] WAVE_NOISE - Apply wavelet denoising. -n [int] WAVE_NOISE - Apply wavelet denoising.
--> Default: None. Subtle: 25. Medium: 50. Strong: 125. --> Default: None. Subtle: 25. Medium: 50. Strong: 125.
@ -96,76 +112,130 @@ OPTIONS, RAW DEVELOPMENT:
--> Option Value: <LS>-<CS>:<LT>-<CT> --> Option Value: <LS>-<CS>:<LT>-<CT>
--> Weak: 2-1:2-3. Medium: 3-2:2-3. Strong: 7-7:5-5 --> Weak: 2-1:2-3. Medium: 3-2:2-3. Strong: 7-7:5-5
--> DONT combine with TEMP_NOISE.
-G [i-i-i-i] REM_NOISE - Yet another spatial denoiser, with 4 choices of 24 modes. -O [i-i-i-i] REM_NOISE - Yet another spatial denoiser, with 4 choices of 24 modes.
--> See https://ffmpeg.org/ffmpeg-filters.html#removegrain for list of modes. --> See https://ffmpeg.org/ffmpeg-filters.html#removegrain for list of modes.
--> Option Value: <mode1>-<mode2>-<mode3>-<mode4> --> Option Value: <mode1>-<mode2>-<mode3>-<mode4>
--> I truly cannot tell you what values will be helpful to you; there are too many... Look at the link above! --> I truly cannot tell you what values will be helpful to you; there are too many... Look at the link!
-g [0:4] SPACE - Output color transformation. --shallow SHALLOW - Output smaller, 8-bit files.
--> 0: Linear. 1: 2.2 (Adobe RGB). 2: 1.8 (ProPhoto RGB). 3: sRGB. 4: BT.709. --> Read why this is a bad idea: http://www.cambridgeincolour.com/tutorials/bit-depth.htm
--shallow SHALLOW - Output 8-bit files.
OPTIONS, COLOR: OPTIONS, COLOR:
-g [0:4] GAMMA - Output gamma. A curve applied to the output, for easier viewing/grading.
--> 0: Standard (Around 2.2). 1: Linear (Default).
--> Requires color-ext: 2: Cineon. 3: C-Log2 4: S-Log3
--> "Standard" grades to the gamut specification, and to 2.2 if that's not given.
-G [0:6] GAMUT - Output gamut. The range of colors that can exist in the output.
--> 0: sRGB (Default). 1: Adobe RGB. 2: Rec.709. 3: XYZ (Always Linear Gamma).
--> Requires color-ext: 4: Rec2020 5: DCI-P3 6: Sony S-Gamut3.cine
-w [0:2] WHITE - This is a modal white balance setting. -w [0:2] WHITE - This is a modal white balance setting.
--> 0: Auto WB. 1: Camera WB (default). 2: No Change. --> 0: Auto WB (requires balance.py). 1: Camera WB (default). 2: No Change.
--> AWB uses the Grey's World algorithm.
-A [i:i:i:i] SHARP - Lets you sharpen, or blur, your footage. -A [i:i:i:i] SHARP - Lets you sharpen, or blur, your footage.
--> BE CAREFUL. Wrong values will give you strange errors.
--> Size/Strength (S/T). S is the size of the sharpen/blur effect, T is the strength of the sharpen/blur effect. --> Size/Strength (S/T). S is the size of the sharpen/blur effect, T is the strength of the sharpen/blur effect.
--> Luma/Chroma (L/C). L is the detail, C is the color. Luma sharpening more effective. --> Luma/Chroma (L/C). L is the detail, C is the color. Luma sharpening more effective.
--> Tip: Chroma blur can actually be a helpful noise reduction technique.
--> Option Value: <LS>:<LT>:<CS>:<CT> --> Option Value: <LS>:<LT>:<CS>:<CT>
--> LS and CS must be ODD, between 3 and 63. Negative LT/CT values blur, while positive ones sharpen. --> LS and CS must be ODD, between 3 and 63. Negative LT/CT values blur, while positive ones sharpen.
--> Strong Sharp: 7:3:7:3 Strong Blur: 7,-3:7,-3 --> Strong Sharp: 7:3:7:3 Strong Blur: 7,-3:7,-3. Medium Sharp: 5:1:3:0
-l <path> LUT - Specify a LUT to apply. -l <path> LUT - Specify a LUT to apply after Color Management.
--> Supports cube, 3dl, dat, m3d. --> Supports cube, 3dl, dat, m3d.
--> Specify -l multiple times, to apply multiple LUTs in sequence.
-S [int] SATPOINT - Specify the 14-bit saturation point of your camera. You don't usually want to. -S [int] SATPOINT - Specify the 14-bit uint saturation point of your camera. You don't usually need to.
--> Lower if -H1 yields purple highlights. Must be correct for highlight reconstruction. --> Worth setting globally, as it's a per-camera setting. Must be correct for highlight reconstruction/unclipped highlights.
--> Determine using the max value of 'dcraw -D -j -4 -T' --> Lower from 15000 if -H1 yields purple highlights, until they turn white.
--> You can determine the optimal value using the max pixel value of 'dcraw -D -j -4 -T'.
--white-speed [int] WHITE_SPD - Samples used to calculate AWB --white-speed [int] WHITE_SPD - Manually specify samples used to calculate AWB.
--allow-white-clip WHITE_CLIP - Let White Balance multipliers clip. --allow-white-clip WHITE_CLIP - Let the White Balance multipliers clip.
OPTIONS, FEATURES: OPTIONS, FEATURES:
-D DESHAKE - Stabilize the video using the wonderful ffmpeg "deshake" module. -D DESHAKE - Auto-stabilize the video using ffmpeg's "deshake" module.
--> You'll probably wish to crop/scale the output in editing, to avoid edge artifacts. --> You may wish to crop/scale the output later, to avoid edge artifacts.
-u DUAL_ISO - Process as dual ISO. -u DUAL_ISO - Process as dual ISO.
--> Requires cr2hdr.
-b BADPIXELS - Fix focus pixels issue using dfort's script. -b BADPIXELS - Fix focus pixels issue using dfort's script.
--> Requires mlv2badpixels.sh.
-a <path> BADPIXEL_PATH - Use your own .badpixels file. -a <path> BADPIXEL_PATH - Use your own .badpixels file. Does NOT require mlv2badpixels.sh
--> How to: http://www.dl-c.com/board/viewtopic.php?f=4&t=686 --> How to: http://www.dl-c.com/board/viewtopic.php?f=4&t=686
-F <path> DARKFRAME - This is the path to a "dark frame MLV"; effective for noise reduction. -F <path> DARKFRAME - This is the path to a "dark frame MLV"; effective for noise reduction.
--> How to: Record 5 sec w/lens cap on & same settings as footage. Pass MLV in here. --> How to: Record 5 sec w/lens cap on & same settings as footage. Pass MLV in here.
--> If the file extension is '.darkframe', the file will be used as the preaveraged dark frame. --> If the file extension is '.darkframe', the file will be used as a preaveraged dark frame.
-R <path> dark_out - Specify to create a .darkframe file from passed in MLV. -R <path> darkframe_output - Specify to create a .darkframe file from passed in MLV.
--> Usage: 'convmlv -R <path> <input>.MLV' --> Usage: 'convmlv -R <path> <input>.MLV'
--> Averages <input>.MLV to create <path>.darkframe. --> Averages <input>.MLV to create <path>.darkframe.
--> THE .darkframe EXTENSION IS ADDED FOR YOU. --> THE .darkframe EXTENSION IS ADDED FOR YOU.
OPTIONS, INFO: OPTIONS, INFO:
-q Output MLV settings. -q settings - Output MLV settings.
-K Debian Package Deps - Output package dependecies. -K [0:3] Dist Deps - Output package dependecies, for use with common package managers.
--> Install (Debian only): sudo apt-get install $ (./convmlv -K) --> 0: Debian, 1: Ubuntu, 2: Fedora, 3: Homebrew (Mac)
--> Deps Install (Debian): sudo apt-get install $(./convmlv.sh -K 0)
--> Deps Install (Ubuntu): sudo apt-get install $(./convmlv.sh -K 1)
--> Deps Install (Fedora): sudo yum install $(./convmlv.sh -K 2)
--> Deps Install (Homebrew Mac): brew install $(./convmlv.sh -K 3)
-Y Python Deps - Lists Python dependencies. Works directly with pip. -Y Python Deps - Lists Python dependencies. Works directly with pip.
-->Install (Linux): sudo pip3 install $ (./convmlv -Y) -->Install (Cross-Platform): sudo python3 -m pip install $ (./convmlv -Y)
-M Manual Deps - Lists manual dependencies, which must be downloaded by hand. -M Manual Deps - Lists manual dependencies, which must be downloaded by hand.
--> There's no automatic way to install these. See http://www.magiclantern.fm/forum/index.php?topic=16799.0 . --> Manually place all in RES_PATH. See http://www.magiclantern.fm/forum/index.php?topic=16799.0 .
COLOR MANAGEMENT:
INTRO Images aren't simple. They are often stored, processed, and viewed as a result of complex transformations usually called,
in applications, color management. Understanding it is required as a colourist, and encouraged as a DPs and Cinematographers.
--> Intro: http://www.cambridgeincolour.com/tutorials/color-management1.htm
--> Understanding Gamma: http://www.cambridgeincolour.com/tutorials/gamma-correction.htm
--> Color Spaces: http://www.cambridgeincolour.com/tutorials/color-spaces.htm
--> Conversions: http://www.cambridgeincolour.com/tutorials/color-space-conversion.htm
--> Monitor Calibration: http://www.cambridgeincolour.com/tutorials/monitor-calibration.htm
PIPELINE convmlv is a color managed application, designed to retain quality from RAW footage:
-- mlv_dump writes camera-specific color transformation matrices as metadata in developed DNGs.
--> This defines the camera's gamut.
-- dcraw applies these matrices, then transforms the newly developed image to the super-wide XYZ colorspace.
--> All color detail is preserved, and if now in a well-defined colorspace.
--> No gamma has been applied - the image is now Linear XYZ.
-- ffmpeg applies the specified (up to) x64 resolution 3D LUTs, in .cube format.
--> Use -g and -G to specify Gamma/Gamut combinations.
--> The output employs DATA, not LEGAL, values. The resulting image is flatter, but retains all shadow/highlight detail.
--> -l specified LUTs are applied afterwards.
3D LUTS The included LUTs, found in the convmlv repository, are key to convmlv's color management solution:
-- color-core: The required LUTs, including sRGB (default), Adobe RGB, and Rec709 in Standard and Linear gamma.
-- color-ext: Optional LUTs, including Rec2020, DCI-P3, etc. in Standard/Linear gammas, but also in Log formats.
Create Your Own LUTs using LUTCalc, for any grading format output: https://cameramanben.github.io/LUTCalc/ (watch his tutorials).
--> Note that convmlv only accepts up to 64x64x64 LUTs. You can resize LUTs using pylut (https://pypi.python.org/pypi/pylut).
--> The pylut command to resize is 'pylut <yourx65lut>.cube --resize 64'. Alternatively, you can use pylut from Python (2X only).
--> I reccommend Legal --> Data LUTs, as this conserves shadow/highlight detail for grading. Legal --> Legal looks better, but with detail loss.
CONFIG FILE: CONFIG FILE:
Config files, another way to specify options, can save you time & lend you convenience in production situations. Config files, another way to specify options, can save you time & lend you convenience in production situations.
@ -175,7 +245,7 @@ CONFIG FILE:
SYNTAX: SYNTAX:
Most options listed above have an uppercased VARNAME, ex. OUTDIR. Yu can specify such options in config files, as such: Most options listed above have an uppercased VARNAME, ex. OUTDIR. You can specify such options in config files, as such:
<VARNAME> <VALUE> <VARNAME> <VALUE>
@ -187,7 +257,7 @@ CONFIG FILE:
CONFIG_NAME <name> CONFIG_NAME <name>
Flags If the value is a true/false flag (ex. IMAGE), simply specifying VARNAME is enough. THere is no VALUE. Flags If the value is a true/false flag (ex. IMAGE), simply specifying VARNAME is enough. There is no VALUE.
OPTION ORDER OF PRECEDENCE Options override each other as such: OPTION ORDER OF PRECEDENCE Options override each other as such:
-LOCAL options overwrite GLOBAL options. -LOCAL options overwrite GLOBAL options.
@ -208,4 +278,4 @@ CONFIG FILE:
Contact me with any feedback or questions at convmlv@sofusrose.com, or PM me (so-rose) on the ML forums! Contact me with any feedback or questions at convmlv@sofusrose.com, PM me (so-rose) on the ML forums, or post on the thread!

93
docs/workflow.txt 100644
View File

@ -0,0 +1,93 @@
The convmlv Workflow is a powerful one.
Shoot Footage -->
convmlv, best-guess preview quality --> convmlv, final quality -->
Blender, VFX -->
Blender, film emulation/Looks --> Blender, color grading -->
Blender/Audacity/etc., Editing
Shoot Footage: Focus on losing no details - we'll be tonemapping later. Dual ISO/HDR isn't bad either. Shoot with correct White Balance, if you can.
--> Ensure that the camera's numbering is not reset while shooting a project. Non-unique filenames will give you headaches.,
Organization: Using MLRawViewer, sort each file by scene, and if preproduction was organized enough, by shot as well.
-footage: Contains resources for development.
raw: Contains all your raw footage. Usually on an external hard drive.
-scn: Scenes of the movie. Contains shot folders or takes.
-sht: Shots in the scene. Contains multiple takes/angles.
conv: Contains all your converted footage.
darkframe_<cameramodel>: Contains darkframe files. Set it up for now.
convmlv: Contains a specific version of convmlv and its dependencies.
<project>.conf
Setup Project: With all the MLV footage organized, it's time to build config files!
FIRST: Ensure convmlv is setup & working correctly from the convmlv folder above.
SECOND: Fill the darkframe_<cameramodel> with darkframes (see convmlv -h) for each combo of ISO, Shutter, Aperture, and Crop/Resolution
in your project.
THIRD: Create a badpixels.txt for your camera.
-Global Config: convmlv isn't always backwards compatible, so make sure you're using a static version of it. Add this VARNAME:
RES_PATH /path/to/footage/convmlv #This makes sure that your project will use one version of convmlv.
-Local Config:
CONFIG_NAME <project>
IMAGE
IMG_FMT 0 #For a Blender workflow, we're working from fast EXR's, not MOVs.
PROXY 3 #We need jpgs; mp4s are more for a quick glance at the footage. 2 is fine too.
PROXY_SCALE 50% #I like a bit smaller proxies
DEMO_MODE 3 #Highest quality, but lower it for preview quality.
SPACE 0 #We're working in Linear space here.
BADPIXEL_PATH <path/to/badpixels.txt> #Interpolate around your camera's dead pixels. They're very irritating otherwise.
Use template.txt in the convmlv repository to organize your config files so far, and add any stylistic options you wish to the Local config.
They will apply to the whole project.
Preview Development: Go through each and every piece of footage, and make a file-specific block in the Local Config that corresponds,
previewing one frame for each/running MLRawViewer to decide. Make sure to add BADPIXELS if your camera is prone to that.
Ask Yourself:
-What frame range do I really need of this?
-Denoising - Do I need it? Is there predictable motion, so temporal denoising is OK? How much detail am I willing to give up?
-Highlights - Should I reconstruct (3-9)? Leave them be (0)? Curve to grey (2)? Allow clipping, setting the SATPOINT too (1)?
-White Balance - Should I let AWB have a go at it, or was the Camera WB fine?
-Deshake - Are subjects small/immobile, making sure the algorithm doesn't latch onto them? Otherwise I might want to do this later, with more precision.
-HDR - Make sure to add DUAL_ISO if you shot HDR.
-LUTs - Add any LUT you want, but know that Blender gives you great flexibility with the looks you want.
-OUTPUT FOLDER - Make sure each file is placed in an OUTPUT directory in 'conv' that corresponds to the file location in 'raw'.
Lower the demosaicing mode (by adding a CLI option), and you're ready! Run 'convmlv -C path/to/local.conf $(find /path/to/footage -name *.[mM][lL][vV])'
to develop everything overnight. Look at the shots in motion the next day, tweak, and you're good to go!
Final Development: If possible, develop it all once for preview's sake. If not, make sure to take it easy on temporal denoising. When your
config file is ready, run (again):
'convmlv -C path/to/local.conf $(find /path/to/footage -name *.[mM][lL][vV])'
Hello, Blender! I like to make one (depending on project size & collaborators) .blend file, import all exr sequences into the Movie Clip Editor (drag the
first exr in the sequence and drop it on the big screen, then press the little "F" so it doesn't dissappear when you save), then make one scene per composite.
Make one scene; name it edit.
MAKE SURE: Scene->Color Management->Sequencer must be set to "Linear".
This is the time to do VFX of any kind. Render to Linear Half EXR's
Film Emulation: Also in Color Management, you'll find some cool options. Render-->View can be turned to Film for film emulation, and various
old-school 3D LUTs can be found in Look. Since we have very nice dynamic range from ML RAW, playing with the exposure/gamma/curves lends us amazing control.
Color Grading: Shift-->A-->Input-->Movie Clip, select the shot named like the scene. Then node away! I like to put a Color-->Tonemap node in front,
as well as a Color Balance, maybe a Lens Distortion; options are limitless.

2
licence
View File

@ -1,6 +1,6 @@
The MIT License (MIT) The MIT License (MIT)
Copyright (c) [year] [fullname] Copyright (c) 2016 Sofus Rose
Permission is hereby granted, free of charge, to any person obtaining a copy Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal of this software and associated documentation files (the "Software"), to deal