Usage: ./convmlv.sh [FLAGS] [OPTIONS] files INFO: A program allowing you to develop ML files into workable formats. Many useful options are exposed. --> Defaults: Compressed 16-bit Linear EXR. 10-bit Prores4444 MOV. --> Color Defaults: Linear (1.0) Gamma on sRGB Gamut, using Camera White Balance. --> Acceptable Inputs: MLV (10/12/14 bit, compressed or not), RAW (requires raw2dng), Folder containing DNGs. --> Option Input: From command line or config file (specify with -C). --> Forum Post: http://www.magiclantern.fm/forum/index.php?topic=16799 . --> A note: BE CAREFUL WITH OPTIONS. Wrong values can give very strange errors. Read this page well!! convmlv is as simple or complex as you need it to be: 'convmlv -m .mlv' is enough for good-looking output! Thank you to the entire Magic Lantern community! This tool merely strings together their tools, without which, convmlv would have been entirely impossible. VERSION: 2.0.2 MANUAL DEPENDENCIES: Place all of these in RES_PATH (see OPTIONS, BASIC). Keep in mind that you will also need dist. and pip packages. --> See 'Dist Deps' and 'Python Deps' -- mlv_dump (@bouncyball's): Required; binary. YOU NEED THIS SPECIFIC VERSION.. --> Only download @bouncyball's version: https://www.magiclantern.fm/forum/index.php?topic=18975.0 . --> IF YOU DO NOT DOWNLOAD THIS SPECIFIC VERSION, YOU MAY BE UNABLE TO DEVELOP COMPRESSED MLVs! -- color-core: Required; folder of LUTs. --> See convmlv repository. -- sRange.py: Required; script. --> See convmlv repository. -- raw2dng: Optional; for DNG extraction from RAW. --> Original forum post w/source code: http://www.magiclantern.fm/forum/index.php?topic=5404.0 . --> Linux users can download from: http://www.magiclantern.fm/forum/index.php?topic=9335.0 . -- mlv2badpixels.sh: Optional; for bad pixel removal. Thanks to @dfort! --> Download from the scripts_archive folder at this link: https://bitbucket.org/daniel_fort/ml-focus-pixels/src -- cr2hdr: Optional; for Dual ISO Development. --> Original forum post w/source code: http://www.magiclantern.fm/forum/index.php?topic=7139.0 . --> Linux users can download from: http://www.magiclantern.fm/forum/index.php?topic=9335.0 . --> Mac users can find the Mac binary in the latest zip: https://bitbucket.org/kichetof/lr_cr2hdr/downloads/ . -- balance.py: Optional; for Auto White Balance. --> See convmlv repository. -- color-ext: Optional; extra LUTs, providing more color management resources. --> See convmlv repository. OPTIONS, BASIC: -v, --version version - Print out version string. -h, --help help - Print out this help page. -C, --config config - Designates config file to use. -o, --outdir OUTDIR - The path in which files will be placed. -P, --res-path RES_PATH - The path in which all Manual Dependencies are looked for. --> Default: Current Directory (.) . --dcraw DCRAW - The path to dcraw. --mlv-dump MLV_DUMP - The path to mlv_dump. --raw-dump RAW_DUMP - The path to raw2dng. --badpixels MLV_BP - The path to mlv2badpixels.sh . --cr-hdr CR_HDR - The path to cr2hdr. --srange SRANGE - The path to sRange.py. --balance BAL - The path to balance.py. --python PYTHON - The path or command used to invoke Python 3. Default is python3 on Linux, python on Mac. -T, --threads [int] THREADS - Override amount of utilized process threads. Default is ALL - 1. OPTIONS, OUTPUT: -i IMAGE - Will output image sequence. -t [0:3] IMG_FMT - Image output format. --> 0: EXR (default), 1: TIFF, 2: PNG, 3: Cineon (DPX)." -m MOVIE - Will output a Prores4444 file. -p [0:3] PROXY - Create proxies alongside main output. --> 0: No proxies (Default). 1: H.264 proxy. 2: JPG proxy sequence. 3: Both. --> JPG proxy will always be in sRGB Gamma/sRGB Gamut. H.264 proxy is color managed. --> JPG proxy *won't* ever be developed w/o IMAGE. H.264 proxy *will*, if specified, be developed no matter what. --> Why? JPG is for potential use in editing. H.264 is for a quick visual preview of convmlv's output. -s [0%:100%] PROXY_SCALE - the size, in %, of the proxy output. --> Default: 50%. -k KEEP_DNGS - Specify if you want to keep the DNG files. --> Run convmlv on the top level folder of former output to reuse saved DNGs from that run! -r - FRAME_RANGE - Specify to process an integer frame range. --> You may use the characters 's' and 'e', such that s = start frame, e = end frame. --> Indexed from 0 to (# of frames - 1). Develops from 1 to ($ of frames) --> If a single # is written, without a -, that frame alone will be developed. --> DO NOT try to reuse DNGs while developing a larger frame range. --uncompress UNCOMP - Turns off lossless image compression. Otherwise, this compression is used: --> TIFF: ZIP, EXR: PIZ, PNG: lvl 0, DPX: RLE. OPTIONS, RAW DEVELOPMENT: -d [0:3] DEMO_MODE - Demosaicing algorithm. Higher modes are slower + better. --> 0: Bilinear. 1: VNG (default). 2: PPG. 3: AHD. -f FOUR_COLOR - Interpolate as RGBG. Fixes weirdness with VNG/AHD, at the cost of sharpness. -H [0:9] HIGHLIGHT_MODE - Highlight management options. --> 0: White, clipped highlights. 1: Unclipped but colored highlights. 2: The detail of 1, but adjusted to grey. --> 3-9: Highlight reconstruction. Can cause flickering. Start at 5, then adjust to color (down) or to white (up). --> Option #2 is mostly the best - the desaturation of highlights is very filmic, and preserves the most detail. -c [0:3] CHROMA_SMOOTH - Apply shadow/highlight chroma smoothing to the footage. --> 0: None (default). 1: 2x2. 2: 3x3. 3: 5x5. --> MLV Input Only. Won't trigger for other input types. -n [int] WAVE_NOISE - Apply wavelet denoising. --> Default: None. Subtle: 25. Medium: 50. Strong: 125. -N - TEMP_NOISE - Apply temporal denoising. --> A: 0 to 0.3. B: 0 to 5. A reacts to abrupt noise (splotches), B reacts to noise over time (fast motion causes artifacts). --> Subtle: 0.03-0.04. High: 0.15-0.04. High, Predictable Motion: 0.15-0.07 -Q [i-i:i-i] HQ_NOISE - Apply Handbrake's 3D denoising filter. --> In depth explanation: https://mattgadient.com/2013/06/29/in-depth-look-at-de-noising-in-handbrake-with-imagevideo-examples/ . --> Spacial/Temporal (S/T). S will soften/blur/smooth, T will remove noise without doing that but may create artifacts. --> Luma/Chroma (L/C). L is the detail, C is the color. Each one's denoising may be manipulated Spacially or Temporally. --> Option Value: -:- --> Weak: 2-1:2-3. Medium: 3-2:2-3. Strong: 7-7:5-5 --> DONT combine with TEMP_NOISE. -O [i-i-i-i] REM_NOISE - Yet another spatial denoiser, with 4 choices of 24 modes. --> See https://ffmpeg.org/ffmpeg-filters.html#removegrain for list of modes. --> Option Value: --- --> I truly cannot tell you what values will be helpful to you, as there are too many... Look at the link! --shallow SHALLOW - Output smaller 8-bit files. --> Read why this is a bad idea: http://www.cambridgeincolour.com/tutorials/bit-depth.htm OPTIONS, COLOR: -g [0:4] GAMMA - Output gamma. A curve applied to the output, for easier viewing/grading. --> 0: Standard (Around 2.2). 1: Linear (Default). --> Requires color-ext: 2: Cineon. 3: C-Log2 4: S-Log3 --> "Standard" grades to the gamut specification, and to 2.2 if that's not given. -G [0:6] GAMUT - Output gamut. The range of colors that can exist in the output. --> 0: sRGB (Default; D65). 1: Adobe RGB (D65). 2: Rec.709 (D65). 3: XYZ (Always Linear; D65). --> Requires color-ext: 4: Rec2020 (D65) 5: DCI-P3 (D65) 6: Sony S-Gamut3.cine (D65) -w [0:2] WHITE - This is a modal white balance setting. --> 0: Auto WB (requires balance.py). 1: Camera WB (default). 2: No WB Scaling of RAW output. --> AWB uses the Grey's World algorithm. Other algorithms are not currently available. -A [i:i:i:i] SHARP - Lets you sharpen, or blur, your footage. --> BE CAREFUL. Wrong values will give you strange errors. --> Size/Strength (S/T). S is the size of the sharpen/blur effect, T is the strength of the sharpen/blur effect. --> Luma/Chroma (L/C). L is the detail, C is the color. Luma sharpening more effective. --> Option Value: ::: --> LS and CS must be ODD, between 3 and 63. Negative LT/CT values blur, while positive ones sharpen. --> Strong Sharp: 7:3:7:3 Strong Blur: 7,-3:7,-3. Medium Sharp: 5:1:3:0 -l LUT - Specify a LUT to apply after Color Management. --> Supports cube, 3dl, dat, m3d. --> Specify -l multiple times, to apply multiple LUTs in sequence. --> LUTs bigger than x64 or smaller than x17 are not supported, and won't apply. -S [int] SATPOINT - Specify the integer saturation point of your camera. Adjust if you're getting purple highlights. --> It's worth setting in a config, as it's a per-camera setting. Must be correct for highlight reconstruction/unclipped highlights. --> How To Use: Start at & lower from a value of 15000 if "-H 1" yields purple highlights, until the highlights turn white. --> This setting is sometimes called "Raw White Point". --> You can determine the optimal value using the max pixel value of 'dcraw -D -j -4 -T'. --white-speed [int] WHITE_SPD - Manually specify the # of samples used to calculate AWB. --allow-white-clip WHITE_CLIP - Let the White Balance multipliers clip. Usually no effect. OPTIONS, FEATURES: -D DESHAKE - Auto-stabilize the video using ffmpeg's "deshake" module. --> You may wish to crop/scale the output later, to avoid edge artifacts. -u DUAL_ISO - Process the footage as Dual ISO. --> Requires cr2hdr. --> Doesn't work for alternating frame HDR Video, where every other frame has a different ISO. -b BADPIXELS - Fix focus pixels issue using dfort's script. --> Requires mlv2badpixels.sh. -a BADPIXEL_PATH - Use your own .badpixels file. Does NOT require mlv2badpixels.sh --> Why? Pixels die on your camera, and show up in footage. This lets you interpolate around these pixels. --> How to: http://www.dl-c.com/board/viewtopic.php?f=4&t=686 -F DARKFRAME - This is the path to a "dark frame MLV"; effective for noise reduction. --> How to: Record 5 sec w/lens cap on & same settings as footage. Pass MLV in here. --> If the file extension is '.darkframe', the file will be used as a preaveraged dark frame. -R darkframe_output - Specify to create a .darkframe file from passed in MLV. --> Usage: 'convmlv -R .MLV' --> Averages .MLV to create .darkframe. --> THE .darkframe EXTENSION IS ADDED FOR YOU. OPTIONS, INFO: -q settings - Output MLV settings. -K [0:3] Dist Deps - Output package dependecies, for use with common package managers. --> 0: Debian, 1: Ubuntu, 2: Fedora, 3: Homebrew (Mac) --> Install Sys Dependencies (Debian): sudo apt-get install $(./convmlv.sh -K 0) --> Install Sys Dependencies (Ubuntu): sudo apt-get install $(./convmlv.sh -K 1) --> Install Sys Dependencies (Fedora): sudo yum install $(./convmlv.sh -K 2) --> Install Sys Dependencies (Homebrew Mac): brew install $(./convmlv.sh -K 3) -Y Python Deps - Lists Python dependencies. Works directly with pip. --> Install Python Dependencies (Cross-Platform): sudo python3 -m pip install $ (./convmlv -Y) --> It may complain about numpy not being found. To fix, just run this first: sudo python3 -m pip install numpy . -M Manual Deps - Lists manual dependencies, which must be downloaded by hand. --> Manually place all in RES_PATH. See Manual Dependencies above. COLOR MANAGEMENT: INTRO Images aren't simple. They are often stored, processed, and viewed as a result of complex transformations usually called, in applications, color management. Understanding it is required as a colourist, and encouraged as a DPs and Cinematographers. --> Intro: http://www.cambridgeincolour.com/tutorials/color-management1.htm --> Understanding Gamma: http://www.cambridgeincolour.com/tutorials/gamma-correction.htm --> Color Spaces: http://www.cambridgeincolour.com/tutorials/color-spaces.htm --> Conversions: http://www.cambridgeincolour.com/tutorials/color-space-conversion.htm --> Monitor Calibration: http://www.cambridgeincolour.com/tutorials/monitor-calibration.htm PIPELINE convmlv is a color managed application, designed to retain quality from RAW footage: -- mlv_dump writes camera-specific color transformation matrices as metadata in developed DNGs. --> This defines the camera's gamut. -- dcraw applies these matrices, then transforms the newly developed image to the super-wide XYZ colorspace. --> All color detail is preserved, and if now in a well-defined colorspace. --> No gamma has been applied - the image is now Linear XYZ. -- ffmpeg applies the specified (up to) x64 resolution 3D LUTs, in .cube format. --> Use -g and -G to specify Gamma/Gamut combinations. --> The output employs DATA, not LEGAL, values. The resulting image is flatter, but retains all shadow/highlight detail. --> -l specified LUTs are applied afterwards. 3D LUTS The included LUTs, found in the convmlv repository, are key to convmlv's color management solution: -- color-core: The required LUTs, including sRGB (default), Adobe RGB, and Rec709 in Standard and Linear gamma. -- color-ext: Optional LUTs, including Rec2020, DCI-P3, etc. in Standard/Linear gammas, but also in Log formats. Create Your Own LUTs using LUTCalc, for any grading format output: https://cameramanben.github.io/LUTCalc/ (watch his tutorials). --> Note that convmlv only accepts up to 64x64x64 LUTs. You can resize LUTs using pylut (https://pypi.python.org/pypi/pylut). --> The pylut command to resize is 'pylut .cube --resize 64'. Alternatively, you can use pylut from Python (2X only). --> I reccommend Legal --> Data LUTs, as this conserves shadow/highlight detail for grading. Legal --> Legal looks better, but with detail loss. CONFIG FILE: Config files, another way to specify options, can save you time & lend you convenience in production situations. GLOBAL: /home/sofus/convmlv.conf LOCAL: Specify -C/--config. SYNTAX: Most options listed above have an uppercased VARNAME, ex. OUTDIR. You can specify such options in config files, as such: One option per line only. Indentation by tabs or spaces is allowed, but not enforced. Comments Lines starting with # are comments. You may name a config using: CONFIG_NAME Flags If the value is a true/false flag (ex. IMAGE), simply specifying VARNAME is enough. There is no VALUE. OPTION ORDER OF PRECEDENCE Options override each other as such: -LOCAL options overwrite GLOBAL options. -COMMAND LINE options overwrite LOCAL & GLOBAL options. -FILE SPECIFIC options overwrite ALL ABOVE options. File-Specific Block: A LOCAL config file lets you specify options for specific input names: / ...options here will only be * You must use the truncated (no .mlv or .raw) input name after the /. Nested blocks will fail. With a single config file, you can control the development options of multiple inputs as specifically and/or generically as you want. Batch developing everything can then be done with a single, powerful commmand. Contact me with any feedback or questions at convmlv@sofusrose.com, PM me (so-rose) on the ML forums, or post on the thread!