Shadowmage

LUA language for Modding

Recommended Posts

Title says it all.

I believe it is a terrible idea from my experience with the language.  LUA is an interpreted language (read: slow to execute, slower to interact with external data and code), has minimal syntax structure (bad for people who like structured code), and is not suited to large or complex projects due to lack of decent IDEs and standard external libraries.  Now, I actually like the LUA language, am decently familiar with it, and have used it quite a bit in the past (for very small projects); but it is that familiarity that leads me to think it is not a good choice for a programming language for mods in a physics-based game -- you want precompiled (or at list JIT'd), as close to the metal as possible.  I could see it being used for actual scripting in-game; e.g. scripting your own autopilot functions per-mission profile; but using it for full mods is like building a house from popsickle sticks -- its just the wrong tool/material for the job.

 

As nobody else seems to concerned with it, I thought I should at least voice my opinion and concerns.  Interested in hearing others' opinions, thoughts, and/or more information regarding why LUA might be a good choice.

Share this post


Link to post
Share on other sites

Accessibility? I found MAS (which heavily relies on LUA) easier to learn than taking a dive into full-on plugins. If this is made according to the same paradigm (LUA+a slew of built-in functions handling KSP-specific things), then the learning curve to plugin programming can be reduced. In-game scripting could really neatly integrate with that.

TBH, I'm actually fully onboard with your concerns. In particular, MAS LUA is slow, but this is not a problem for things that execute once per button press/switch flip. I try not to use LUA calls for things that update continuously. It doesn't perform well enough to implement something like SSTU, MAS or Principia, and it's downright silly to expect the game will both have all features we'll want, and that they will be implemented in a competent way.

Share this post


Link to post
Share on other sites
Posted (edited)
32 minutes ago, Shadowmage said:

Title says it all.

I believe it is a terrible idea from my experience with the language.  LUA is an interpreted language (read: slow to execute, slower to interact with external data and code), has minimal syntax structure (bad for people who like structured code), and is not suited to large or complex projects due to lack of decent IDEs and standard external libraries.  Now, I actually like the LUA language, am decently familiar with it, and have used it quite a bit in the past (for very small projects); but it is that familiarity that leads me to think it is not a good choice for a programming language for mods in a physics-based game -- you want precompiled (or at list JIT'd), as close to the metal as possible.  I could see it being used for actual scripting in-game; e.g. scripting your own autopilot functions per-mission profile; but using it for full mods is like building a house from popsickle sticks -- its just the wrong tool/material for the job.

 

As nobody else seems to concerned with it, I thought I should at least voice my opinion and concerns.  Interested in hearing others' opinions, thoughts, and/or more information regarding why LUA might be a good choice.

@Snark Says he believes that Non-Lua mods will be allowed, just like they are now. 
 

 

Edited by GoldForest

Share this post


Link to post
Share on other sites
16 minutes ago, Dragon01 said:

Accessibility? I found MAS (which heavily relies on LUA) easier to learn than taking a dive into full-on plugins.

I can accept that argument for what it is; valid point.  LUA scripting certainly is easier to get into than learning a full language + environment + all of the tools.

 

16 minutes ago, Dragon01 said:

I try not to use LUA calls for things that update continuously. It doesn't perform well enough [...]

Indeed -- its fine for occasional scripting, something that runs every few frames, or every few seconds, or by occasional event/trigger.  But I would never to do physics calculations with it, heavy math, or even anything more than a line or two that had to execute every frame.

(apparently, since the last time I've used it, LUA has JIT compilers available...which could put it much closer to on-par with C# from a speed perspective;  still not fond of its lack of strong syntax/structure for larger projects)

 

1 minute ago, GoldForest said:

@Snark Says he believes that Non-Lua mods will be allowed, just like they are now. 

That would make sense, and could actually be a functional improvement over what we have now.  Hopefully someone can confirm.

 

The LUA end would be a built-in KOS sort of thing, for basic scripting tasks, which could interact with the KSP API and any exposed mods' APIs; and leave the heavy lifting and complex calculations to c# or c++ native code.

Share this post


Link to post
Share on other sites
8 minutes ago, Shadowmage said:

The LUA end would be a built-in KOS sort of thing, for basic scripting tasks, which could interact with the KSP API and any exposed mods' APIs; and leave the heavy lifting and complex calculations to c# or c++ native code.

The phrasing they gave is somewhat ambigous, and could even be taken to mean tools themselves were written in LUA, though that would be a very silly thing to do. :) I could definitely get behind something like kOS and/or MAS being built in from the start, as long as it's not a replacement for proper plugins.

Share this post


Link to post
Share on other sites
Posted (edited)

i don't think i would use lua, its stagnated quite a bit. i love the language it just lacks good repositories. the system ksp has is a lot more versatile. you pretty much have direct access to the whole damn engine. with lua you would have to write middleware for everything you want it to have access to. c# is just better and requires a lot less code to implement. 

now if you want a programming language you can use in game id go with some kind of graphical programming language to do some light vessel scripting. 

Edited by Nuke

Share this post


Link to post
Share on other sites

According to Scott Manley, Lua will be entirely optional, and will serve a purpose similar to what ModuleManager does now: an easy way to change things without complex code. C# modding will be fully supported as always.

Share this post


Link to post
Share on other sites

I never understood the game industry's fascination with Lua. It's not a bad language, it's just... weird. I've been learning Python lately, which I think is a much better language with a lot of current libraries.

3 minutes ago, coyotesfrontier said:

According to Scott Manley, Lua will be entirely optional, and will serve a purpose similar to what ModuleManager does now: an easy way to change things without complex code. C# modding will be fully supported as always.

Judging by the below comment, that seems to be correct. Lua looks like it's set up as a supplementary modding tool, not for core functionality. As an aside, this is similar to how it's been used on some other games. World of Warcraft allows user-customizable UI with Lua, and way back when I worked on Ultima Online we used Lua internally for creating dialog windows for quests and such. I'd still prefer Python, but you can't have everything I guess.

Quote

I'm pretty sure that plug-ins will still be just good ol' C#, same as they've always been.

The only reference to Lua that I saw was as an input to a cool scripting tool that they'd built, which not only allowed them to automate the game in interesting ways (e.g. for repro cases and such)... but the mere fact that this tool was running outside the game, and talking to the game through some interface, and that they'd put a lot of refactoring work into the game to make it support that interface, and that anyone else could write a tool to talk to that interface... told me that they really take modding seriously.

TL;DR:  I'm pretty sure that all the ways modders currently mod will continue to mod pretty much the same way.  But there may be some additional new goodies that let us do stuff we couldn't easily do, before.  There may or may not be some Lua involved in that stuff.

 

 

Share this post


Link to post
Share on other sites
5 hours ago, Shadowmage said:

I can accept that argument for what it is; valid point.  LUA scripting certainly is easier to get into than learning a full language + environment + all of the tools.

 

Indeed -- its fine for occasional scripting, something that runs every few frames, or every few seconds, or by occasional event/trigger.  But I would never to do physics calculations with it, heavy math, or even anything more than a line or two that had to execute every frame.

(apparently, since the last time I've used it, LUA has JIT compilers available...which could put it much closer to on-par with C# from a speed perspective;  still not fond of its lack of strong syntax/structure for larger projects)

 

That would make sense, and could actually be a functional improvement over what we have now.  Hopefully someone can confirm.

 

The LUA end would be a built-in KOS sort of thing, for basic scripting tasks, which could interact with the KSP API and any exposed mods' APIs; and leave the heavy lifting and complex calculations to c# or c++ native code.

i found the biggest constraint in lua was its memory consumption. having wrote a good fraction of a game engine in pure stand alone lua, it handles my totally optimized collision detection code just fine, but try to load a texture and lol it grinds to a hault as the gc has to deal with a lot of temporary data. and it doesnt store it effitiently. if you wanted to store r,g,b elements in a table with named keys, expect to use a ton of memory. using indexed values didnt help much either. i just ended up storing the data in strings which gave a lot more performance. the correct lua way to deal with this is to create a userdata type set up in a module or host application that was coded with c. 

Share this post


Link to post
Share on other sites
26 minutes ago, sturmhauke said:

I never understood the game industry's fascination with Lua. It's not a bad language, it's just... weird. I've been learning Python lately, which I think is a much better language with a lot of current libraries.

Python and nearly all others were designed as fully stand-alone programming languages.  Lua was designed as a language for embedded add-ons - not something you write the whole game in, but something you could bolt in quickly and let users work in.  There aren't many languages written for that niche.  (And to reply to Nuke's comment: In that niche, repositories would be fairly useless - you're not supposed to be writing large segments of code in it, and you should be writing code working against specific other applications' APIs, so large libraries of code aren't really going to be helpful for what it's designed for.)

Share this post


Link to post
Share on other sites
43 minutes ago, sturmhauke said:

I never understood the game industry's fascination with Lua. It's not a bad language, it's just... weird. I've been learning Python lately, which I think is a much better language with a lot of current libraries.

Judging by the below comment, that seems to be correct. Lua looks like it's set up as a supplementary modding tool, not for core functionality. As an aside, this is similar to how it's been used on some other games. World of Warcraft allows user-customizable UI with Lua, and way back when I worked on Ultima Online we used Lua internally for creating dialog windows for quests and such. I'd still prefer Python, but you can't have everything I guess.

 

 

i kind of feel that way about python. lua is very similar to other languages ive used. but python has a very strong community, well maintained repos, and uses beyond embedding it in a c program. lua would be great if it had that kind of community backing it.

Share this post


Link to post
Share on other sites
Posted (edited)

Lua is what they are using during development.  There is nothing which locks them to Lua, and it's too early for them to lock anything like that.

And Python has it's own oddities

Edited by linuxgurugamer

Share this post


Link to post
Share on other sites

I'll admit, the more I work in Python, the less I like it.  But what do I know?  I'm a Perl fan.  ;)

Share this post


Link to post
Share on other sites
9 hours ago, Shadowmage said:

As nobody else seems to concerned with it, I thought I should at least voice my opinion and concerns.  Interested in hearing others' opinions, thoughts, and/or more information regarding why LUA might be a good choice.

Based on what I've seen thus far, you don't have to worry.  This isn't a thing.  No, mods aren't "going to be LUA now".  Unless someone else has a source of information that I don't-- which I suppose is possible, but seems unlikely to me and I'd really like to see a citation for where all this is coming from-- then this is all a misunderstanding.  My guess as to what has happened is that someone took some small comment out of context somewhere and then extrapolated from that.

8 hours ago, GoldForest said:

Says he believes that Non-Lua mods will be allowed

It's not that they're allowed, it's that mods aren't Lua and I believe there are no plans to make them so.

The whole Lua thing is something else entirely.  Explanation below, but the TL;DR is that no, it's not what you'll use to write plugins (that's still C#), and no, I don't believe it's what you'd do with configs and stuff for parts and the like, either.

So what is it?

[EDIT]  Correction.  What I originally wrote here was close, but not quite spot on.  I've since had a chance to chat with a Star Theory engineer to clear up exactly how, why, and for what they're using any Lua, and it's much clearer now.  See my post below for details.  However, the gist-- which is "you won't need Lua to write mods"-- is still correct.

What I originally wrote here is preserved for posterity, in spoiler.

Spoiler

Here's my original explanation, which is not quite correct based on information received after I wrote it.  So please ignore the following unless you just like watching me be somewhat wrong.  ;)

Here's the deal:

They wanted to add an external (i.e. external to the KSP executable) interface that allows automating the game from outside of it.  Use case is things like setting up repro cases for bugs, etc.  So they've given it some sort of interface (not sure if it's RPC or a local network socket or whatever-- I don't have any technical details about the exact mechanism), which will be accessible and (hopefully) documented and anyone can write a tool that drives it if they are so inclined.  They wanted this for their own use cases and testing purposes (it's a lot easier to test the game if you can programmatically set up situations and things), but realized that it would also be useful to modders and such who could conceivably really go to town on something like that.

And this was no small investment on their part.  When they realized they needed it, they did a major refactor of the code to support this; it was a significant investment of time and energy for them.  So it's a first-class citizen.

Then, having done that... they went and built a tool (i.e. a separate executable, not part of KSP.exe itself) that can talk to that interface.  They demoed that tool to us.  And that tool happens to be written to accept Lua as input, so you can have Lua scripts to do things like "okay, now move the Mun, and add another moon here.  Oh, take this spaceship and move it over there so it'll be on an intercept course to the Mun."  And like that.  Think of it as HyperEdit on steroids, with a scriptable interface and access to basically everything in the game's state.  They used Lua in that particular tool because Lua's kind of convenient for scripting and was a well-known quantity.

Now, what I don't know?  I don't know if that tool is actually something they'll release publicly for people to use, or whether it might just be some internal testing tool for their own consumption.  But even if they're not releasing it... it doesn't have any special "in" with the game, i.e. there wouldn't be anything stopping some enterprising modder from producing their own test harness to talk to the same interface that this test tool was talking to.  I could imagine, for example, that interface being super useful for making something like Telemachus.  :)

Anyway.  So that's the story with the Lua thing.  AFAIK, that's the only actual place where Lua has entered the conversation, so my assumption is that all the rest of this speculation and concern has been sparked by that, in which case everyone can stop worrying, because "mods will be Lua" isn't a thing.

 

If anyone does have some other source that this speculation is stemming from-- which I'm guessing is not the case, but hey, I could be wrong-- please provide a citation because if I'm wrong, I'd like to know.  ;)

Share this post


Link to post
Share on other sites
31 minutes ago, Snark said:

My guess as to what has happened is that someone took some small comment out of context somewhere and then extrapolated from that.

Pretty much exactly that.  The only mention that I saw regarding technical details of modding was a statement about 'modding tools are being written using LUA' (paraphrased from memory...).  Trying to find the specific post that mentioned it, but not having much luck at the moment (it was a compilation of notes/transcript from some videos or interviews that was compiled by another forum-user...but don't remember their name offhand either).

Which sparked this whole thing off.  LUA has its purposes.. but plugin authoring is not one of them.  I thought it was an extremely odd decision and course of action for them to take, which is why I opened this thread for further discussion.

 

If we can be reasonably confident that plugins will still be authored using c#, then I would say this thread can be closed, as there is nothing really to cause concern.

(I'm perfectly fine with some LUA based tools being included, esp. the sort that you mention regarding external (or internal) automation tools -- that is exactly what the language was meant for, and a perfectly suited case for its use)


Thanks for the input and info :)  (and feel free to close/lock this thread to not continue discussing false info and speculation)

Share this post


Link to post
Share on other sites
34 minutes ago, Shadowmage said:

Pretty much exactly that.  The only mention that I saw regarding technical details of modding was a statement about 'modding tools are being written using LUA' (paraphrased from memory...).  Trying to find the specific post that mentioned it, but not having much luck at the moment (it was a compilation of notes/transcript from some videos or interviews that was compiled by another forum-user...but don't remember their name offhand either).

What was said was that they are currently using tools written in Lua for the game development, nothing about modding using Lua

Share this post


Link to post
Share on other sites

Okay, an update.  After writing the above, I actually bumped into a Star Theory lead engineer who was hanging out near their booth, so I had a chance to quiz him a bit about the Lua thing so I could get it straight from the horse's mouth.

What I wrote above is not far off, but it's not quite spot on.  Here's some clarification / correction / additional info:

They've written the game so that it does, in fact, have a Lua interface, built into the game.  Pretty much anything you can do with a C# mod, there's Lua access as well.  He explicitly stated that there is nothing you can do in Lua that you couldn't do with C# code, which means you never actually have to write Lua if you don't want to-- it's not any sort of gatekeeper.  So the debugging tool I saw them using wasn't making the choice of Lua there-- what it was, was a handy tool for streaming Lua into KSP's Lua interface.

The game (currently) supports Lua in config files.  That doesn't necessarily mean that "all config is Lua now"-- they're not saying yet what their standard config format will be.  But it supports it, so you can do stuff there.  What makes Lua cool as a config language is that you can put logic and callbacks and stuff into it.  He gave, as an example, a case where he wanted to add a special right-click menu item for a particular part in order to try something out, but didn't want to have to build a full-fledged mod around it by writing in C#, compiling, the whole nine yards.  So he could just slip some Lua logic into the config and accomplish some very mod-like behaviors right there with no additional tools needed.

The other reason why they're motivated to enable that is dynamic loading.  C# is a compiled language, which means if you want to change anything (e.g. iterating during debugging), you have to recompile the code, exit the game, restart the game, try it out.  Whereas Lua's interpreted, and they (currently) have some hooks in place that allow dynamically loading Lua while the game's already up and running so that they can tweak and tinker and rapidly iterate without having to go through a bazillion cycles of restarting the game.  The actual game code's all in C#, but often they need to rapidly iterate to try stuff out, and that is something this allows.

The engineer was careful to stipulate that this is all still a work in progress, and that he was describing to me what they have now, during development, specifically to support development.  It's not clear how much of this will be left in place when they ship.  For example, the hot loader that lets Lua get swapped out dynamically at run time, or the ability to have Lua callbacks in config, that stuff-- it may or may not stay, or may change around radically.  So don't take this to the bank yet, there's still a long road until release.

Anyway, hopefully this clarifies things.  :)

TL;DR:

  • Lua may (or may not) be present and available to do super cool things with, at release time.
  • Lua isn't (and won't be) mandatory in any way, and you can always write mods in C# same as you've always done.  That's a definite.

Share this post


Link to post
Share on other sites
Posted (edited)

the dynamic loading thing is a great debugging tool. in my engine you can drop the console and put it into lua mode. and write code for the game while its running. so if i need to spawn an object or change my position in the world i just type in the correct function call. it doesnt do it yet but i could easily make it so it would save the command buffer to a file and let you load the same code with another call. even if this kind of thing doesnt exist in ksp2 you could easily implement it via mod. you still want c# for any performance critical code though. 

did they say what version they were using. i mostly still use 5.1 because a lot of the libraries i use don't work in newer versions and havent been updated. but 5.3 has some nice features. 

Edited by Nuke

Share this post


Link to post
Share on other sites
10 hours ago, Nuke said:

did they say what version they were using. i mostly still use 5.1 because a lot of the libraries i use don't work in newer versions and havent been updated. but 5.3 has some nice features.

None of us happened to ask them, so no, they didn't mention it.

Share this post


Link to post
Share on other sites
Posted (edited)

Being able to write in both C# and lua is probably about as good a pair of options as we'll get. Lua is light and simple to support, and not being compiled gives it complimentary uses to C#.

Python isn't really fit to purpose in my eyes. It's slower than lua, and it has a bigger footprint. The advantages it would enjoy are chiefly that it's very popular (which may open up the door to some potential modders), and that it is backed by very comprehensive standard libraries. I wouldn't complain if they supported Python because its a language I really like (it's also possibly the easiest language to read, which makes it perhaps the best language in which to be tinkering with lightly documented code written by other people :P), but it wouldn't be at the top of my list.

Edited by Randox

Share this post


Link to post
Share on other sites

a seldom used feature of lua is that you can compile down th bytecode if you want to protect your sources. 

Share this post


Link to post
Share on other sites
On 8/31/2019 at 2:00 AM, Snark said:

The other reason why they're motivated to enable that is dynamic loading.

Loud grumbling.

On 8/30/2019 at 10:53 PM, DStaal said:

Python and nearly all others were designed as fully stand-alone programming languages.

While I don't like Python (perhaps because I'm paid to write it - like every 2019 hipster developer I want to write Rust and/or Typescript, but I digress and long compilation times are coffee breaks) it was actually very much built to be embeddable. In fact Sims 4 is moddable in Python. 

it's also possibly the easiest language to read

Extremely loud grumbling.

Edited by ModZero

Share this post


Link to post
Share on other sites
4 minutes ago, ModZero said:

Loud grumbling.

Because?  Why is this not a good thing?

Grumbling tends to be more helpful if it comes with a "why" or a "what would you like better" along for the ride.  ;)

On 8/31/2019 at 1:07 PM, Randox said:

Being able to write in both C# and lua is probably about as good a pair of options as we'll get.

Was pretty much my own take on it as well.

Share this post


Link to post
Share on other sites
7 minutes ago, Snark said:

Because?  Why is this not a good thing?

I oppose the thesis of the sentence, not imply anything about whether it's good :-) You don't need an interpreted language for dynamic loading and/or unloading, nor having that automatically enables dynamic loading. You need some sort of dynamic linker (or a thing that fulfils the same purpose within the context), and if you want unloading, well-defined cleanup procedures. You can, and often have, dynamic loading with code written in C and C++ - that's how old browser plugins worked, that's how Linux, MacOS and Windows load device drivers (note to self: check how Windows does that now, they might have went all microkernely). I think having a scripting interface is great because it let's you have a built-in REPL prompt (REPL doesn't really work well for most compiled languages, they're not designed for that sort of thing and it gets unwieldy if you try), but that's different from dynamic loading.

7 minutes ago, Snark said:

Grumbling tends to be more helpful if it comes with a "why" or a "what would you like better" along for the ride.  ;)

Sorry, it was obvious in my brain --' and I didn't want to take space with a huge paragraph. I didn't expect the "it's not a good thing" interpretation.

Also, the forum didn't let me enter an emoji, I'm sad now.

Share this post


Link to post
Share on other sites
8 minutes ago, ModZero said:

I oppose the thesis of the sentence, not imply anything about whether it's good :-) You don't need an interpreted language for dynamic loading and/or unloading, nor having that automatically enables dynamic loading. You need some sort of dynamic linker (or a thing that fulfils the same purpose within the context), and if you want unloading, well-defined cleanup procedures. You can, and often have, dynamic loading with code written in C and C++ - that's how old browser plugins worked, that's how Linux, MacOS and Windows load device drivers

Yes, but for a compiled language like C#, at minimum you at least have to recompile.

They're clearly super busy trying to get things working, over there, so I think it's pretty safe to assume that they wouldn't take the time to build a thing unless they were pretty darn sure that the payoff outweighs the cost of building it.  From chatting with the engineer who was describing it, it was clear that having an interpreted language that didn't need recompiling was extremely valuable to them for rapid iteration-- it's nimble.  Also, for certain very simple types of logic, the fact that it's lightweight (i.e. "just insert a few lines of text here", without having to go through the entire process of implementing and building a C# assembly to house the mod) can also be a win.

 

Share this post


Link to post
Share on other sites
This thread is quite old. Please consider starting a new thread rather than reviving this one.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.