[WIP][RFC] Kerbcam 2

[huin]

First up - no promises. I've made attempts to work on Kerbcam in the past, and have found myself not delivering - at least in part because of a lack of energy/enthusiasm/time and a codebase that got nasty and hard to change. So the intent is for a rewrite, leveraging the lessons learnt from the original Kerbcam, and maybe selected code, but leaving the bulk of it behind.

What I'm doing right now is experimenting with ideas for a rewrite. Even if I never execute on this, then maybe it'll be of use to someone else that might want to do this. This post is intended to be an RFC (request for comment), as it seems like a bad idea to continue too far into this without feedback, particularly from potential users.

The design so far is living on Github. I'll paste in the raw markdown here, but emphasize that the design in this post is likely to remain as an initial revision and might not get updated even if I update the version on Github. If aspects of the design aren't clear to potential users, then that's a bug, and things should be clarified.

I intend for the core features to be relatively simple and achievable, but leaving the overall structure (see the story/timeline/etc. concepts in the design) to be a good starting point for building on for more advanced features.

The design: (warning - might be out of date, see the original source)

## Concepts for Kerbcam2

### Story

Top-level container object for a timeline, its tracks, and operations.

### Timeline

Contains a set of points in time relative to a "time zero" start point. These
points in time *may* be named.

### Time key

(Could there be a better name?)

A point in time (within a timeline) that may be referenced by operations. It
may or may not be named for convenience.

### Track

Not a functionally required concept, but potentially useful for the user to
organise operations into groups within a timeline purely to serve a
user-interface need to present and organise a sequence of operations clearly. A
constraint being that operations that overlap in time cannot exist within the
same track. An operation exists in exactly one track (maybe zero tracks in a
transitional state if there are copy/paste mechanics).

A track may or may not be named for convenience.

### Operation

Something that manipulates the camera (or potentially part of the scene) as
time progresses. This uses points in time referenced from the timeline. An
operation may be instantaneous, or cover a range of times.

Examples of operations:
* Interpolate camera zoom (field-of-view).
* Interpolate camera position.
* Interpolate camera look-at point.
* Control time speed.
* Invoke an action on a part (e.g activate an engine).
* (Maybe) Force GC cycle for perfectionists - maybe this point in the sequence
  isn't too (subjectively) sensitive to GC pauses, but a part that *is*
  sensitive is coming up.
* (Maybe) Set camera visible layers. (This should allow for a IVA view)

Operations *might* have sub-operations in future, but that's just a
speculative feature idea for now.

### Actuator

A general set of interfaces for something that acts as a proxy for modifying
the camera or scene. This is distinct from modifying the object(s) directly so
that the actuator can:

* Cleanup after playback.
* Reduce the number of places that Kerbcam interfaces with KSP's own API, thus
  limiting the impact of breaking changes in KSP.
* Be used as a stub to create preview visualizations. (E.g a stub object that
  implements a camera actuator can record camera position etc. over time and
  then be used to render an object showing how the camera moves)
* Potentially be used as a fake for unit testing actuators etc.
* (Maybe - future speculation) Further use in layering operations. E.g a
  camera looking direction operation transitioning to another by a
  interpolation might delegate to both looking-direction sub-operations and
  smoothly interpolate between the results on stubbed cameras and apply that to
  another actuator that applies to the actual camera.

Actuators should exist to support for the various types of operation that
exist.

## Misc thoughts

Time.captureFramerate - potentially useful to set a framerate if we're going
to capture at a given rate, might make the framerate more predictable.

Create lights to move around a scene as an operation.

## User Interface

Mockups made using [ASCIIFlow](http://asciiflow.com/).

Open questions:
* How should things be grouped into windows? For example, might the "main
  window" be arranged into 3 sections:
  * Playback controls.
  * Story overview.
  * Item properties (for editing an item selected in the story overview).

### Playback controls

* Simple "Play" toggle.
* Auto-play setting that causes playback (i.e time "zero" on a timeline) to
  begin at a given UT (game universe time). This UT time setting maybe should
  be part of the data for the story.

### Story overview

This mockup shows how the timeline UI might look with a reasonably simple
story.


              +-------+-------+-------+---------------+--------+--------+
              | 0.00" | 3.15" | 7.50" | 10.23"        | 12.73" | 20.28" |
              | Start |       |       | Engines start |        |        |
    +-----------------+---------------+---------------+--------+--------+
    |         |               |                                         |
    | Track 1 | Camera pos... | Camera position                         |
    |         |               |                                         |
    +-------------------------------------------------------------------+
    |         |               |                                         |
    | Track 2 | Look at po... | Look at position                        |
    |         |               |                                         |
    +-------------------------+-------+---------------+-----------------+
    |         |                       |               |
    | Track 3 |                       | Invoke        |
    |         |                       |               |
    +---------+                       +---------------+

Discussion of each component in this mockup:
* The top row is the timeline, containing the time keys. A couple of them are
  named, but they all exist at specific times, and are kept in order of
  increasing time from left to right.
* Each row below that is a track, in this example they just have default names.
* Each track row contains operations. In this example:
 * "Track 1" contains operations that move the camera, the first operation is a
   movement of the camera position starting at 0.00", and finishing at 7.50".
   Immediately following that, the camera starts from a new position at 7.50",
   and continues moving until 20.28".
 * "Track 2" is similar to "Track 1" but contains camera "look-at" operations
   instead of camera positioning.
 * "Track 3" contains a single operation at 10.23". This operation (like the
   others) has a default name. It has been configured to invoke the "activate"
   action on some rocket engines on a vessel.

UI decisions made by this mockup (not necessarily set in stone):
* Things can optionally be named (specifically "time keys", "tracks" and
  "operations").
* The column widths are determined by the time key names, but not by the
  operation names. Not clear if this is a good idea or not, or if it's
  practical within Unity3d.

### Path editing

When editing/viewing paths (particularly camera positions, look-at positions),
the path should be visible by the flight camera, so its position can be seen
relative to the environment/vessel. The nodes should also be visible (and
ideally clickable to select). The position of each node should be adjustable
via controls in a window. A node's timing is determined by a referenced time
key. Minimal controls would include X,Y,Z adjusters.

Ideally the position of each node would be adjustable via something akin to the
manuever node interface that KSP players are familiar with. While this seems to
be possible, that's regarded as a stretch/later feature (not a high priority).

A node's position (or the entire path) should be relative to a reference frame.
Examples of reference frames:

* Selected vessel/part/object
 * including rotation
 * ignoring rotation
 * reference position relative to object's position, plus some multiple of the
   object's velocity vector relative to the SOI (rotating/non-rotating SOI).
   Questions remain open as to when the position and velocity are sampled. The
   intent is to help with fly-by effects, but some clarity is needed here.

Edited December 6, 2014 by huin
Update design code

[glen.mack]

First off, while I fully accept - "No promises" I am super excited at the concepts;

I'll only comment on the things that I fully understand and feel need comment.

-Timeline: I'm not sure if this is covered in something later that I can't quite wrap my head around with layering operations. But here's what would be immensely useful.

One of the main problems I have with doing VFX and having more than one thing moving on screen at a time is lighting (due to scaling planets, solar system etc. angles change faster.) If you do two shots of the same thing consecutively they will have noticeable lighting differences. The best I've done so far involved counting off mission times down to the second and even then it was somewhat unreliable. Being able to link "time zero" to either mission time or KSP's internal clock would make a world of difference in this regard. (Note not sure how many other people go to these lengths, lol)

The mission countdown protocol was used to bring five shots of the same thing together:

-Operations: Invoking actions would be a major MAJOR improvement to my life in videos. At this point people are talking about using mods that let you fly multiple craft in videos and I have enough keys to be tapping to be panic inducing as it is. Being able to turn on/off anything in the screen would be huge. My question is, is it limited to being an on/off switch? Would we be able to set flaps? To have Kerbals plant flags/take samples? (Dreaming here I know, but linked to key times, this could lead to some incredible videos.)

Everything else is beautiful. At least what I understand. I'm very confused by Actuator section. Probably due to my lack of understanding of programming.

I wish you all the best in the endeavor Huin. I will certainly be available to test and pass comment and help if need be.

[huin]

Timeline

I don't think it's going to be feasible for Kerbcam to be able to rewind/replay the universe state. In the limited KSP filming I did I tended to use quicksave/quickload quite a bit to achieve that rewind, so I imagine that's how a video maker would do it. Am I correct in understanding that the following might be a typical workflow?

• Create a story with time keys, one of the time keys initiates an action 5 seconds after playback starts, and you want that action covered from multiple camera angles.
  
• Quicksave shortly before UT 5:10:2:32 (5 days, 10 hours, 2 minutes, 32 seconds into a new game)
  
• Set the story timeline to "auto-play" at UT 5:10:2:32. Record the sequence.
  
• Copy the story with a different camera setup. Record the sequence.
  
• Record as many other camera angles as you want.
  
• Use the multiple camera angles of the event in video editing.
  

By quick-saving shortly before the timeline and actions take place, there is hopefully a reduced window during which minor differences happen between playbacks that turn into something noticeable (i.e reducing the "Butterfly Effect"). The auto-play timer takes care of the rest.

That said, there might be another feature you're asking for (having looked at your video), which might record control actions given to a vessel and play them back (and have a few of these to multiple vessels). No idea how feasible that is - although the timeline/operation model certainly sounds like it would fit into the overall architecture design. I wonder if there are other mods that might be hookable for this, or if other mod authors have ideas how that might work.

Operations

An operator is a fairly general concept, and I don't see any reason why it couldn't do things like set flaps, throttles, etc. - it's likely just a matter of taking the time to implement it (assuming the APIs are there, and I imagine at least some of them are). Again - it looks like that fits the architecture.

Actuator

The "actuators" concept isn't something I'd worry about overly from a user perspective, come to think of it. That's something of an internal design decision that should avoid some problems that plagued the original Kerbcam.

[One-Way Films]

+rep for you. Following. This will come in handy if this works. Good luck

[glen.mack]

[*] Set the story timeline to "auto-play" at UT 5:10:2:32. Record the sequence.

This I think this works perfectly, an "auto-play" button would take care of the issue. It would vastly improve the continuity of multiple shot videos as well as sort my overlay issue.

[Darren9]

Unless I'm doing something wrong I haven't been able to run a Kerbcam path with TimeControl slow motion, the Kerbcam clock overrides it. Either a compatibility with slowed game time or similar time/physics control in Kerbcam would be good.

[huin]

Unless I'm doing something wrong I haven't been able to run a Kerbcam path with TimeControl slow motion, the Kerbcam clock overrides it. Either a compatibility with slowed game time or similar time/physics control in Kerbcam would be good.

Ah yes, I've heard that that feature doesn't work. I'll add the "control time speed" feature to the list of example operations.

[Drew Kerman]

IVA camera control would be great, one of the things missing from the current Kerbcam

[huin]

IVA camera control would be great, one of the things missing from the current Kerbcam

Agreed. It was already in the design as a maybe, although wasn't obviously worded as such without knowing KSP internals. The "camera visible layers" is the thing that would do that. I've clarified that a little.

[huin]

So one revision to the design right now is the tracks concept. I think that I'll be removing that concept and make the UI display exactly one operation per row below the timeline. This simplifies a number of things from a development perspective, and I have come to the opinion that it would only force users into a fiddly situation of annoying restrictions as they change things having to manually bin pack operations into tracks - which would suck.

[philotical]

CT is very easy to use, but KC was/is more to my taste..

I appreciate that this does not die..

Especially I like the ability to save camera paths and reuse them.

if you mean that with "tracks", don't remove them completely please..

Sorry, not sure about your terminology.

PS: I'm intrested in this project, so if you feel, I could assist in any ways, PM me without hesitation..

[Halban]

At the moment I'm happy with Kerbcam. Although if I could add something I'd definitely go with a built in zoom function similar to the one used in Camera Tools - it's really nice to be able to get a more cinematic FOV sometimes. I'd also consider changing the manual camera controls so that instead of individual buttons, it used a digital joystick type system. Similar to what they use for movement in a lot of mobile touch screen games. Oh by the way, I'd be (really) happy to test kerbcam 2 in the field of battle for you whenever you need it - just to see how it does when it comes to actual film making.

Edited December 10, 2014 by HatBat

[Engineer of Stuff]

KerbCam could defiantly use an upgrade.

[huin]

Not to worry, tracks wouldn't be connected with saving and restoring paths. Tracks were intended to be a user interface element to group operations together onto the same row in the user interface.

The thing that would be saved and restored would be the "story" concept. I haven't put saving and restoring on the design doc, but there aren't major blockers to having it.

[huin]

At the moment I'm happy with Kerbcam. Although if I could add something I'd definitely go with a built in zoom function similar to the one used in Camera Tools - it's really nice to be able to get a more cinematic FOV sometimes. I'd also consider changing the manual camera controls so that instead of individual buttons, it used a digital joystick type system. Similar to what they use for movement in a lot of mobile touch screen games. Oh by the way, I'd be (really) happy to test kerbcam 2 in the field of battle for you whenever you need it - just to see how it does when it comes to actual film making.

Zooming and dynamically changing zoom is already on the design doc.

Interesting, I hadn't thought of that sort of control system. It's not likely something I'd do in a first version, but maybe something to incrementally add later. The fiddly thing with a touchpad (?) style control is that it's inherently two dimensional, and another axis needs to be available. The particularly interesting point that I think you're suggesting is making it an analogue control rather than digital. Does that interpretation make sense?

[Halban]

Not entirely, I'm not very literate in this area of tech - but yes, I think that's what I'm suggesting. For camera movement through 3D space, normal systems could be used for left, right, backwards and forwards, but a button could switch the mode so that forward with the virtual joystick corresponds as up and backwards as down (left and right would be non-responsive). Although personally I can't see myself using this function for camera movement, rather the panning of the camera (for settings keys, not live work). I'll make a rendition of what I imagine this system would look like.

Edited December 10, 2014 by HatBat

[Halban]

Here it is:

In this example the user is dragging the centre node to the right causing the camera to pan right continuously until the user drags the centre node back to the centre or lets go of the mouse button.

Javascript is disabled. View full album

[huin]

Yep, I think that's what I thought you meant. I don't know how easy that is to do with Unity's GUI library components - they're a little bit... Interesting, shall we say?

Whilst the design as it stands doesn't rule out camera panning, the initial idea for "pathing" the camera's direction is to have it look at a particular position. The original KerbCam worked by interpolating the camera orientation (via "slerping" - spherical linear interpolation), which had some jarring effects when transitioning from one "segment" (i.e the time between keys) of the sequence to another, and it also didn't track objects terribly well (which I suspect was the primary use case for directing the camera). There was a better mathematical model to interpolate smoothly, but I burnt a lot of time prior to the first KerbCam 1 release trying to work out how to do it, but eventually failed and just released something which felt "good enough", but which I was never happy with. Of the KSP cinematics I've watched that seem to use KerbCam, I think that they often avoided the effect by using only a single path segment.

I'm hoping (at least in the early versions) to improve on things by tracking two independent positions in the world:

* The camera's position (smooth interpolation of that was pretty much an easily solved problem in KerbCam 1).

* The position that the camera is looking at.

Both positions would be pathed with the same mechanics.

It's quite possible to add new types of operation later on that would orient the camera by angles instead of looking at a position. There are some papers on "smooth quarternion interpolation" when one does a websearch on that topic, which I (if I'm feeling supremely confident) or someone else with more 3D math credentials, could implement.

[glen.mack]

Please tell me you haven't given up on this Huin!

[Jiraiyah]

common, don't give up, you rock sir, i am waiting for a great news from you

[burnoutforzai]

Don't give up ;(

[EdwardB3020]

Can you make Kerbcam for 1.3 please? I've been wanting this mod for this mod for a long time and soon!

[technicalfool]

It looks like this thread has not had any new information in quite some time. I'll lock it for now, and @huin is welcome to ask a moderator to unlock it if and when an update is due.