The Internationalization Project

[Majiir]

I've been thinking for a couple weeks about supporting internationalization (language packs and more) for mods. It's not an area in which I have much expertise, so I'd like to open this as a community project. I have a few goals in mind:

1. Enable mods to support other languages and cultures. (Duh.) Whether you know it or not, your mod is probably used by a lot of people whose primary language is not English. Once I started looking, I was pleasantly surprised at how many international KSP communities use Kethane and discuss it in their native language.
   
2. Put the fire under Squad to internationalize KSP. If mods can achieve this through the more restrictive KSP API, surely the game itself can do it too. In principle, large parts of the stock game can be internationalized like a mod, but some things are hard-coded.
   
3. Get modders thinking about internationalization concerns. Some programmers are already thinking ahead, but many hardcode strings or turn off culture-specific number formatting without understanding or caring about the consequences.

To those ends, I'd like to build a system for modders that abstracts away the difficulties of internationalization. Here are some problem areas:

• Natural language in config files. Part names and descriptions are clearly in this category, but there are also cases that can't as easily be handled like module-specific parameters. Modders will require a flexible system for replacing language strings in a variety of configuration files.
  
• Natural language in plugins. Some kind of API needs to be available for substituting language strings in plugins, which often generate graphical interfaces or other text.
  
• Selecting a language (and other user concerns). Users need to install language packs and select their language of choice with as little difficulty as possible. Since users are by definition people who can benefit from using a language other than English, it's critical that these processes be intuitively understood.
  
• Displaying alternate character sets. Not all parts of KSP support rendering characters for other languages, but in some cases we can build workarounds. This will involve a lot of diving into the UI and building replacements. Luckily, the same component (EZGUI) that seems to lack support for alternate characters is also easy to manipulate from plugin code.
  
• Internationalization of stock game systems. This is a more general description of the previous problem. The game and many of its components were not built with internationalization in mind, but support might be added by manipulating or replacing things like part modules, UI elements, et cetera.
  
• Translating language strings. In my opinion, this is the easy part. There are large number of multilingual players, and translation tasks can easily be distributed to many people and improved over time.

I'll be working on this as time and other projects allow. In the meantime, I'm interested in your feedback on the above, your thoughts on how to proceed, and ultimately your participation in making this project a reality. Thanks!

[finkmac]

I think the correct word is "Localization", but I may be wrong…

*checks wikipedia*

Hmm, it says both words work… But I usually see "localization" in computer programs.

Your call, I guess

[Majiir]

Internationalization is the process of making some software support multiple languages and cultures. Localization is making that software support one specific language. In this case, my main concern is internationalization so that localization becomes accessible to anyone.

[asmi]

.NET runtime has built-in support for internationalization through the use of sattelite assemblies. It only requires that all localizable resources (not just strings, but any other resources) are put into .NET resource files (*.resx). Then by creating a copy of this resx file with language identifier (for example, if your "language-neitral" resource file is called LocalResources.resx, you create a copy named LocalResources.fr.resx - for French, LocalResources.ru.resx for Russian, etc), translating resources inside these localized versions, and compiling all these.

If you want I can create a sample project to demonstrate this approach.

I'm also working on small GUI library for KSP projects that is XML-based and provides WPF-like interface, the library itself is almost complete and I'm working on creating samples and testing it out. I was thinking about integrating internationalization support into this library, and looks like I need to do just that

Edited September 9, 2013 by asmi

[Majiir]

.NET runtime has built-in support for internationalization through the use of sattelite assemblies.

Yes, and this may be useful for some plugins, but I argue it's not the best solution for a few reasons:

1. Someone has to build the assemblies, and this substantially raises the barrier to entry for anyone who wants to provide localized content. (They could hand off their resources to someone else to build them, but that potentially introduces a bottleneck and it's a pain in the butt for everyone involved.)
   
2. It only provides resources for plugins. This is acceptable for plugin-only mods, but if you also want to localize config files, you're now distributing multiple files in a language pack and distributing those resources in two different formats.
   
3. Localized content can't be shared between plugins and config files, leading to duplicate entries (which is never good).

Moreover, there are larger problems with KSP i18n than simply loading the strings. I welcome discussion here, but my inclination is to say resource files aren't the way to go.

Here's loosely what I have in mind at this point, obviously subject to modification:

• Plugin API for retrieving localized content by key
  
• Config file syntax for injecting values by key, and a plugin to perform the injection
  
• Language files mapping keys to resource values

The advantage here is that all localized content is in the same format, and that data can be shared between both mechanisms (and even a third if it becomes necessary). It's a little indirect, but that indirection is intentional and will prove helpful in some cases.

[Bartybum]

I have no experience with coding whatsoever, but I could help you with Polish, if needed. Also, are you talking about a pre-made set of sentences/words/phrases, or are you talking about a dynamic system based on numerous grammar patterns?

[stupid_chris]

Well afaik, Squad is planning to make KSP available in other languages, but it's down the development line as the UI is too much prone to change and would require it to be retranslated every time. I'm not sure how much support for changing the language of the stock UI is currently available if someone wants to do it manually though. It would be neat if someone could do it via a modulemanager file.

[Majiir]

Also, are you talking about a pre-made set of sentences/words/phrases, or are you talking about a dynamic system based on numerous grammar patterns?

Translations will be manual. (I am taking a course in computational linguistics, but I'm not that good. Machine translation is hard.)

Well afaik, Squad is planning to make KSP available in other languages, but it's down the development line as the UI is too much prone to change and would require it to be retranslated every time.

Egh, I think this is a crappy excuse. Squad doesn't have to actually perform any translations because the community can create language packs. The issue is that Squad has shown no interest in adding even primitive support for localization. Repeating what I said in my first post, getting text translated is the easy part; having a way to get that text into the game is hard, and it's what requires our attention.

I'm not sure how much support for changing the language of the stock UI is currently available if someone wants to do it manually though. It would be neat if someone could do it via a modulemanager file.

Only a small handful of things (like part descriptions) can be changed. I tried earlier today to build a system for localizing resource names (e.g. Kethane -> 케테ì¸) and found that it is possible, even if arduous, to replace some stringsâ€â€but EZGUI won't render other character sets. There's no easy way forward here, so we'll have to hack our way to language support.

[stupid_chris]

Well if you believe so, you better contact the right people about it. I believe they don't plan on the community making the translation for the game itself. Although some more support for changing the game's text through mods might be made available.

Once the game is closer to completion and the texts are more or less stable' date=' SQUAD will probably start working on translations. At this stage of development, it would be more worth than it's worth as changes would need to be done for each language for each update/patch.[/quote']

Source

Edited September 9, 2013 by stupid_chris

[popeter45]

i think its called localisation rather that Internationalization

if you need welsh and or funny fake languages i may be able to help

[m4v]

Currently, in Linux, launching KSP with a locale other than English thoroughly trashes the game, dunno what KSP does in windows as I don't have to force it to English there, but until this is fixed i18n in Linux is probably broken.

How do you usually do i18n in C# code anyway? From my experience in open source projects all I know that for i18n you use gettext, but that isn't probably true for most stuff done in Windows.

[Kimberly]

It's English, it is internationalized. I agree that "localized" is the word you're looking for, Majir--because what you're looking to do is appeal to local groups and their languages, rather than the world as a whole. Waste of time, if you ask me.

[m4v]

It's English, it is internationalized.

Is not. I would argue that is "localized" to English, if it were internationalized then it would be easy to localize to other language, like, if KSP already had the English strings in .po files or whatever ready for translation.

Waste of time, if you ask me.

I wouldn't call it "waste of time" but I think is better to wait until Squad start working on this. Edited September 9, 2013 by m4v

[Bartybum]

i think its called localisation rather that Internationalization

if you need welsh and or funny fake languages i may be able to help

We MUST include Klingon. No excuses

[stupid_chris]

Come on people are we really debating on what the term should be? Let's just drop it and say it is now "interbananization".

No but seriously, could we get some actual info and help on that, I'm interested to see how I could try helping and translating KSP to French.

[Faark]

1. Natural language in config files. Part names and descriptions are clearly in this category, but there are also cases that can't as easily be handled like module-specific parameters. Modders will require a flexible system for replacing language strings in a variety of configuration files.

No. Just no. Don't even know where to start why this is wrong.

- An average user should never have to dig in your CFGs. I never did for Kethane. If they have to, that means your doing it wrong. Add a damn settings menu that ofc has localized labels, descriptions and validation!

- If they really want to modify your config data, that means they have to do some forum research on what those values mean anyway. That doc should be sufficient even for non-english guys.

- This would mean that you cannot use the international (english) doc to edit your local config, and local docs usually are much rarer... if there are any, at all!

- Have you any serious example of someone localizing their config files? Your windows registry for sure is not.

Imo you are overkilling it. Especially with

Internationalization of stock game systems. This is a more general description of the previous problem. The game and many of its components were not built with internationalization in mind, but support might be added by manipulating or replacing things like part modules, UI elements, et cetera.

It's not possible for a lot of stuff, any you would most likely have to redo / redistribute large parts of original game code. Part cfgs? Names should not be changed, again. Sure, changing descriptions doesn't hurt. You have to make a lot of changes that might conflict with other mods or updates. For me it looks like spending a loooot of time, just to push squad a little bit. And i would prefer pushing squad towards making the game run smoother, for now^^

Anyway... yeah, it would be nice to localize your mods to a bunch of languages and allow users to add their own. Some examples other mod developers could rely on would be nice, as well as a single centralized language selection system across all supporting mods that is easy to integrate but does not conflict with other mods. Btw... getting and maintaining a good level of quality for your translations is a hard job. Just as a little warning...

Edited September 9, 2013 by Faark

[asmi]

@Majiir:

Using shared resources among different mods needs to be very careful as some things could have difference meanings in different contexts. Not to mention what would happen if one mod dev would decide to change it for whatever reason. I'd keep them separate just to be safe.

As for localizing non-plugin parts, in can be done in a manner similar to what ModuleManager does. We could override stock part descriptions with it and come up with some sort of special syntax that would instruct MM to look up localized string. It could also provide API for plugins to do the same.

There is one problem though - this all is applicable to strings only, but there might be cases where other resources would need to be localized as well, so we'll need to think about that as well.

Regardless of how this will do, I would offer my assistance in this project. I'm professional .NET developer with over 10 years of experience in this field, and although I generally don't have as much spare time as I wish I'd have, I will do my best to help out.

[Majiir]

Currently, in Linux, launching KSP with a locale other than English thoroughly trashes the game, dunno what KSP does in windows as I don't have to force it to English there, but until this is fixed i18n in Linux is probably broken.

I'll check it out on my Linux VM. It's still possible to work around if we're willing to lose culture-specific number formatting.

How do you usually do i18n in C# code anyway? From my experience in open source projects all I know that for i18n you use gettext, but that isn't probably true for most stuff done in Windows.

Resource files are commonly used as asmi described, though there may be commonly used libraries.

No. Just no. Don't even know where to start why this is wrong.

Let's start with the part where you completely misunderstood what I wrote. I was referring to natural language values in config files. For example, the KethaneWetMassIndicator module has a Label property which changes the info panel string from "Wet Mass" to whatever's specified. Extraplanetary Launchpads uses this because "wet mass" is an odd term to use for a container of rocket parts. Whenever a natural language string is in a config file, it'll need to be replaced.

It's not possible for a lot of stuff, any you would most likely have to redo / redistribute large parts of original game code.

A whole lot can be done (with varying degrees of difficulty). Obviously we can't modify everything, but we can certainly tackle some important parts of the UI. This has a cascading effect in some cases: if I can't change how the UI displays resource names, then I can't translate "Kethane" into other languages without causing confusion. A few key spots in the UI can make a big difference.

Using shared resources among different mods needs to be very careful as some things could have difference meanings in different contexts. Not to mention what would happen if one mod dev would decide to change it for whatever reason. I'd keep them separate just to be safe.

I agree. What I wrote was that resources should be shareable between components in the same mod, so e.g. I can translate "Kethane" once and use that string in both my plugin and part configs. Cross-mod references aren't a horrible thing if they're namespaced, but that seems like complexity with little benefit.

As for localizing non-plugin parts, in can be done in a manner similar to what ModuleManager does. We could override stock part descriptions with it and come up with some sort of special syntax that would instruct MM to look up localized string. It could also provide API for plugins to do the same.

Indeed. I'm eying ModuleManager as a good candidate for modification as a component of this system.

There is one problem though - this all is applicable to strings only, but there might be cases where other resources would need to be localized as well, so we'll need to think about that as well.

Support for textures, sounds and binary files should be sufficient, and that can come in a more polished version. In a worst-case scenario, those assets can be swapped out directly in GameDatabase.

[UbioZur]

Just for info of why a game is not localized before the end of the development:

- an average of professional translation cost .2$US per word per language. To that you have to add the testing time (translator do not usually have access to the game, so you need a specialized international testing team) to make sure the translation make sense. if you spend that money early it's a waste of money because as the game get developed, you will need to keep paying to get the change translated, so it's a waste of money, and it's better off developing a game in one language and then pay for the translation once.

However since, SQUAD have plan to localized KSP, I hope they are using either .net or unity or in house tools to localized strings. and if it's the case, we may have a way to plug ourself on it.

as of an example:

the Command pod MK1 cfg would cost US$10.6 per language, or (if we consider a language per international forum), US$63.6 for one craft file. or more than 3 people buying the game (even more as we should remove any tax, steam% and all that stuff... and the testing time)

another example the .20 ASAS would have cost: $92.4

and with the .21 change add: US$57.6 (the manufacturer is the same).

to get a better view of it, if you have all the game versions, get a idea of how much each update would have cost (don't forget the time that is needed to send file for translation, get it back and test it), get your hourly income, and see how many hours of work it would have been wasted on localization instead of actual game content.

As for what can be done at the moment:

- anything we have access to (I.e. .cfg files) are easy to translate. and using something like Module Manager.

- I believe lot of the GUI text are in the program and pre-compiled, so it will be hard to change them.

- trying to have the mod developer to use the same system to localize. providing them with a API even? (like a quick class file that would allow them to manage the different languages)

If in need to help for french translation, let me know.

[Capt'n Skunky]

We have every intention of adding other languages. As has been pointed out, this is one of the tasks you save for the end of development. Do you want to add another week to each release as we go back and make sure any text that has changed is properly updated into all the languages supported? It would require time that is better spent elsewhere and can be saved by waiting until all the text is close to finalized.

Bottom line, yes, we will add other languages, but it's not going to happen until the end of development.

The issue is that Squad has shown no interest in adding even primitive support for localization.

That statement is flat out wrong Majiir and you know it.

Cheers!

Capt'n Skunky

KSP Community Manager

[Greys]

I think we need to forbid the topic of translations.

As said in the original post, translations are the easy part, what's need is the infrastructure in the game and in the mods to permit translations more easily.

Module Manager may permit you to swap values in parts, but it's kind of a big pain and very manual and there's stuff it cannot do.

- You cannot have some character sets

- You cannot have multiple alternate languages installed

- You cannot change some things, such as the name of a resource; without an external program to parse and replace the resource names in everything including game saves (breaks compatibility between instances without an annoying correction process)

- All managing of languages must be done outside the game by editing and moving files

- Module Manager can replace bits of .cfg files, but it cannot interface with addons, meaning you'd need to have separate systems

Edit:

Here's my suggestion for how to support language sets in .cfgs

LANGUAGE_DEFINITION{
	// Defines a language to be available throughout the rest of the program
	// By doing it this way anyone can add their language or fake languages as desired, and it'll all be tied together by languageKey
	languageKey = EN
	title = English
	description = Some text in the language to help it be clear what language it is, should probably be the same text in all languages
	keys = that represent text fields accross stock KSP
	KSC = Kerbal Space Center
	VAB = Vehicle Assembly Building
	etc = and so on
}

RESOURCE_DEFINITION{
	name = LiquidFuel
	density = 0.005
	flowMode = STACK_PRIORITY_SEARCH
	transfer = PUMP
	LANGUAGE{
		languageKey = EN
		author = Greys // Author of this language block
		title = Liquid Fuel // KSP will use title instead of name in UI, so name never needs to change, protecting compatability
	}
}

PART{
	lots = of stuff

	LANGUAGE{
		languageKey = EN
		author = Greys
		title = Wing
		description = It makes you fly!
		manufacturer = Collumbus
	}

	MODULE{
		more = stuff
		LANGUAGE{
			keys = that are useful to the module in question
		}
	}
}

This will permit multiple language sets to be included and distributed with every part, Module Manager can be used to edit them into stock configs as needed, and the cfg can pass stuff to plugins. It's still up to the plugin to use it, but it's available.

Depending on how the code works, this could also be used to swap texture sets based on language to have native text on parts.

I'm not much of a programmer, but if LANGUAGE{} could be set up so that the LANGUAGE{languageKey} block that matches that set for the game as a whole passes all the keys that it doesn't use itself up to it's parent, that'd probably do something.

Edited September 9, 2013 by Greys

[Majiir]

We have every intention of adding other languages. As has been pointed out' date=' this is one of the tasks you save for the end of development. Do you want to add another week to each release as we go back and make sure any text that has changed is properly updated into all the languages supported? It would require time that is better spent elsewhere and can be saved by waiting until all the text is close to finalized.[/quote']

You (and many others) are still changing what I'm saying to mean something else. This is about adding support for localization, also known as internationalization or i18n. That is, the act of internationalizing KSP is the same as building in support for localization. The two words don't mean the same thing. Never did I say Squad should translate KSP into other languages and waste a whole lot of development time. I'm a software engineer, I understand how this works; internationalization is absolutely something that you want to add early in the development process. It becomes much harder if you wait until later. Contrast this with localization, which you correctly observe is difficult to do during active development.

So, when I say that Squad has shown no interest in internationalizing KSP, I think I'm justified. The game lacks basic support in many components. Even if Squad never published translations, there's no way for anyone else to do it. Plus, whenever someone brings it up (this isn't the first time), they get the same canned "we have to do that at the end" response.

As said in the original post, translations are the easy part, what's need is the infrastructure in the game and in the mods to permit translations more easily.

Greys is right on with the reading comprehension.

Module Manager may permit you to swap values in parts, but it's kind of a big pain and very manual and there's stuff it cannot do.

ModuleManager is licensed CC-BY-SA, so we can tweak it for our purposes. I'm mainly interested in the more internal config-replacing components. ModuleManager-like syntax could be very useful for specifying which pieces of a config need to be mapped to language keys, but for language strings themselves I suggest using a different format. Config nodes have issues with escaping values (and since this has been on their bugtracker for a long time, I don't imagine it'll be fixed any time soon).

[kotysoft]

I'm not a native speaker, but I understand what is the difference between "localization" and "internationalization" as Majiir said before.

Just to clarify: I have no any programming skill, and don't know how hard is to call text to a compiled program from a simple text file, but for example, Sixteen Tons Entertainment made it, and working well.

For example: Take a look on EMERGENCY 4's easily scriptable language structure >> DOWNLOAD ZIP 8.1MB

(Cutted for smaller size and to avoid licence problems, just left some "sample" things inside)

If you look inside the compressed file, all of their fonts are in editable DDS files and you can modify it easily if you want additional special characters or acutes. (if I remember well, attached one already edited to hungarian acutes)

And take a look on an opensource PHP webshop "software"s language structure (opencart) HERE 44kB

Ok. Calling strings in PHP is MUCH easier than in C#... it's just for example.

I bet if SQUAD making a free-editable format for language files, and let us access the "sample" english one (what game using), the following languages will appear within a month or less under CC-BY-SA licence:

German, Russian, Spanish, French (by Chris ), Polish, Czech, Hungarian, and maybe Italian.

(ok, there are many more languages but I mostly see these ones have nice translator communities)

---

EDIT: maybe a little help for plugin coders about XML handling in unity: http://www.andrewnoske.com/wiki/Unity_-_XML

---

EDIT2: Relating to part.cfg files imho it's better to leave them untouched and consider all of them as default English language.

A "simple" plugin maybe can handle part descriptions and titles. in my idea a plugin have a simple ingame GUI where you can force SCAN/UPDATE of available files and a dropdown menu to select your language. Basically a plugin have own config files for languages and for their specifications, like i posted in another thread:

code		= 'EN'
name		= 'English'
direction	= 'ltr'
dateformat	= 'd/m/Y'
decimal		= '.'
thousand	= ','

Once you select your language in this plugin's GUI, then it's forcing to override original lines of PART.CFG files by possible localized strings. For example if you set hungarian in plugin, then it's enforcing and overrides "title=" ; "description" parts of original cfg file by *-HU.lng (part-HU.lng for example).

Since you can make more different cfg files for a single model in same directory, an individual language file possibly not enught.

LNG files should be pretty easy and short:

title		= 'Money making space-printer'
description	= 'A simple part for printing money on your space station'

The structure looks like:

MRCHEATER / PARTS / PRINTER /

printer.mbm

printer.mu

part1m.cfg

part2m.cfg

/ LANG /

part1m-HU.lng

part2m-HU.lng

part1m-DE.lng

part2m-DE.lng

etc

Edited March 22, 2014 by kotysoft

[Javster]

Apologies for the bump, but I think this is a very important matter: adding other languages to a game becomes harder as you add more features.

Prison Architect (a game I use for many examples) has added Steam language pack integration. This means that the devs do the 1% of the work which enables the community to do the 99% as they need.

Squad don't seem to like Steam, but it doesn't have to be through Steam.

[edvardm]

I agree with localizing not being waste of time. While I prefer to use any software in lingua franca of computing, there's at least one thing: very young kids, who have just learned to read (and still learning).

The point is, I'm teaching stuff to kids (say, 6+ years) with KSP, and one of the biggest issues is that everything is in English.

Good thing is that whether they like or not, they are now learning English at the same time a bit, but sometimes I feel like it's a bit too much in addition to all that math, physics and stuff