Lumiera
The new emerging NLE for GNU/Linux

1. Requirements & Considerations

It seemed easiest to me to just go through the workflow chronologically

(a) Ingest

i.Technical

All ingest sources need a solid metadata structure, suggestions welcome:

  • Media Type attributes (codec, frame mode/rate, aspect, color science/LUT)

  • Clip Attributes (name, length, I/O, media I/O, bitrate, size

  • Acquisition Attributes (Modified EXIF data, ie: format, frame mode/rate, aspect, resolution, ISO, f-stop, shutter speed focal length, color format, camera,)

  • UID & Project/Sequence & Subclip & Render Reference

  • Production Info (scene #, shot #, angle #, take rating, night/day, production name, DP, Camera Op, Director, set/location, date shot, actors/characters)

Tape/Video Ingest

Connect to capture system through back-end/modules/plug-in. Deck-control needs to handle both Firewire (1394) and Serial (RS232, RS422, Sony, Panasonic, JVC) could be internal controller, but at least a timecode hand-off to a plug-in. Needs to be able to control decks in single frame increments with precise TC.

Ability to mark the source tape it came from in the metadata, with the ability to reconnect a when later rerunning the ingestation step. Integration with batch capture

Question: would it make sense just to integrage +dvgrab* ?

Data Ingest

Extract/Write metadata to file module needed. Should parse files to make sure they’re not corrupted

ii.User

  • Should we look into set metadata collection systems to see if modules to sync and copy that data to the clip is viable?

  • The GUI Ingest element should be logical and convenient to use, and in my opinion should handle both tape-based logging-capturing and data ingest.

(b) Edit

i.Technical

A. Video & Audio playback

Requirement-1: Engine should handle at least 4k, 32-bit, 200+MB/s, selectable LUTs (?) Real-time of native codecs is important. Engine should handle in-sync minimum 8 audio tracks, 96kHz/24bit

  • Support for extended colour space, no built-in hard limits (→ Gimp is limited to 8bit)

  • “offline” capability

Requirement-2: Main media playback windows, clip selector & timeline output both should meet Req-1 Live scope simulators, vector and waveform a must, 3d/wheel good choice Live playback TC readouts from playheads

Requirement-3: Compatibility for low-latency hardware output to video & audio devices, w/timecode, likely a hand-off but would latency is important. - Integration to JACK, ALSA, or OSS would be ideal. - Real-time rendering of simple filters: support OpenGL - Background CPU rendering

B. File Handling

Other apps (FCP) are unable to or inconsistently work with updated media files mid-session, this is a back-end design concern.

C. Project/Sequence Requirements

  • Project bins, user-created bins, media filtering (ex: all video or everything in scene #1)

  • Thumbnail & list views

  • Drag & Drop of clips between bins, playback windows & timelines.

  • File and directory import

D. Timeline

All the normal essentials

Render indicators (needs render, RT) on clips and/or timeline Should be adjustable (frame mode/rate, would require script to translate old TC numbers to new timebase)

I recommend that the timeline video format setting be the “box” and clips “fill in” the box. FCP for example does weird things to the clips when you change the resolution/aspect of the sequence. Further it would make sense to have options in the sequence for how to handle non-native resolutions and aspect ratios (stretch, scale w/boxes, leave alone)

We have Track Nesting, but we need to have some way to gang sync/lock video with its audio track so both are cut and move simultaneously.

Currently, NLEs all use vertical, additive layering. We should work on including the ability to “flowchart” tracks (AE’s pick-whip) like a compositor would, and change the “blend” modes.

Draw-on automation/keyframes & editing for both video & audio tracks.

Note
comment by Ichthyo 5/2015

After careful consideration, we decided for Lumiera to abandon the usual metaphor of a “Track”. One of the benefits is that the problems mentioned here become irrelevant

  • since we don’t have audio vs video tracks, we don’t need to “synchronise” artificially what is connected by its very nature. If media contains video and audio tracks, all that data can be used within the same context.

  • since we build a node graph automatically, instead of “pumping” data through tracks, metaphorically speaking, we gain a much more flexible approach to layering. Basically we have a layering rule, which includes an overlayer mode (“blend mode”). This global rule can be modified in any given scope, down to wiring a single element deliberately to another destination.

  • as an extension to the mentioned routing and overlaying rule, we’ll allow to set up routing rules triggered by media tags.

E. Misc

  • We need to have accessible, easily scriptable FX/Filter menus for clips, all NLEs throw a tab into the clip selector pain.

  • Whole app should be multi-core aware, especially any rendering engines.

  • Needs to work on multiple monitors

  • auto save / auto log operations

ii.User

A. The 6 things an editor thinks about at work:

  • The clips in the bin

  • Timeline

  • Timeline output

  • The edit tools

    • slice / roll / slide / slide through

    • move selection

    • fill

  • Transitions

  • Filters/FX

    • no need for gimmick filters

    • matte filters, colour keying

    • automatable mask

B.Features that make life better

  • Edit review/edit on the fly (Avid)

  • Quick Transitions (1-2 key/click transitions)

  • Match-frame

  • Media Search

(c) Export/Finishing

i.Technical

EDL export

Should export all major EDL formats

XML

We should look into something along the lines of the Apple XML, so that the project could be passed to XML compliant programs

Media Export

Need to build a module that will render project and possibly re-encode it as well, but that could be a hand-off

Tape Export

Essentially the reverse of the Ingest issue, must pass frame accurate deck control and fully synced audio/video.

Finishing

I’m not sure if they plan to integrate color grading/finishing into Lumiera at some point but if not we need to look into compatibility for those applications.

ii.User

EDL/XML Export

We need to leave the workflow that passes on lists in standardized forms. These also need to be easy enough to use without some application console script.

Media Export

Whether handled internally or externally we need to build an interface to make this seamless and intuitive, my favorite exporter is Premiere’s which will render the output with any given setting.

Tape Export

Should be able to handle both straight lay-off and “edit-to-tape”.

Batch export of multiple projects with multiple selectable output formats at once.

2. Types of Workflows

(a) Film

  1. Captured Film-TK-Sound Sync

  2. Ingest-NLE-

  3. Three possible output formats

    1. EDL-Optical Print+Neg Cutting

    2. DI (Either new or oringinal TK)-Film

    3. DI (Either new or oringinal TK)-Video/Media