[API] Open Node Parser

[Ippo]

Open Node Parser is a small utility class that can parse .cfg files and return a ConfigNode object. The advantage is that ConfigNode.Load() can only be called from assemblies that are running inside KSP, while this parser can be used in any application.

The source code is on github. To use it, just include that file I linked in your project.

License is CC-0, which basically means "do whatever you want".

[NathanKell]

That sounds great!

You know, it'd be pretty awesome to hook this up to MM, so one did not have to load KSP to see the changes MM does to nodes...

[Ippo]

Uhm... I think the hard part would be to run MM actually

There are a lot of other "forbidden" methods, and one of them is Debug.Log. If MM calls it, you crash (and I'm not even gonna check, I'm *sure* it's called somewhere).

[taio]

Found a logic bug. The parser assumes cfgs have a strict format that (for example) MechJeb’s MechJeb2_AR202/part.cfg does not conform to. That file has four top‑level nodes, where your parser assumes one. Also, where your parser assumes braces are always on their own lines, that file has several child nodes on one line, e.g.:

MechJebModuleCustomWindowEditor { unlockTechs = flightControl }

There are other mod parts that don’t have braces on their own lines, though I haven’t noticed any that had > 1 top‑level node.

[Ippo]

Not a bug then, just very limited behaviour

I think it's an easy fix, I might give it a shot tonight.

[taio]

I’ve been writing my own in Julia for another project, so here’s another format issue you’ll run into: apparently, leaving off a “}†at the end of the document (i.e. leaving the last top‑level node unclosed) is tolerated. And a bunch of the stock cfgs do this. Sigh…

[Ippo]

Dammit squad -.-

[taio]

LOL Squad… Ran into something else I thought I’d share. Some of the cfgs are in a bizarre encoding. Wrote this Python script to investigate (after removing all mod files):

import chardet
import fnmatch
import os
from collections import Counter

path = 'C:/Program Files (x86)/Steam/SteamApps/common/Kerbal Space Program/GameData/'
result = {}
c = Counter()
for root, dirnames, filenames in os.walk(path):
    for filename in fnmatch.filter(filenames, '*.cfg'):
        filepath = os.path.join(root, filename)
        x = chardet.detect(open(filepath, "rb").read())
        result[filepath] = (x['encoding'], x['confidence'])
        c[result[filepath]] += 1
print(c) # frequency table
bad = []
# find all non-ascii characters by file, row, and column
for (k, v) in result.items():
    if v != ('ascii', 1.0):
        bad.append(k)
        print(k[76:], v) # strip beginning of path
for file in bad:
    print(file[76:])
    for col, line in enumerate(open(file, encoding='utf-8')):
        for row, ch in enumerate(line):
            if ord(ch) >= 128:
                print('\t[{0}:{1}] {2} ({3:4x})'.format(row, col, ch, ord(ch)))

It gives this output:

Counter({('ascii', 1.0): 266, ('UTF-8-SIG', 1.0): 6, ('ISO-8859-2', 0.8515969760557984): 1})
Squad\Parts\Utility\ladderTelescopic\ladderTelescopic.cfg ('UTF-8-SIG', 1.0)
Squad\Parts\Utility\ladderRadial\ladderRadial.cfg ('UTF-8-SIG', 1.0)
Squad\Parts\Command\advancedSasModuleLarge\advSasModuleLarge.cfg ('ISO-8859-2', 0.8515969760557984)
Squad\Parts\Utility\ladderTelescopicBay\ladderTelescopicBay.cfg ('UTF-8-SIG', 1.0)
Squad\Parts\Science\GooExperiment\gooExperiment.cfg ('UTF-8-SIG', 1.0)
Squad\Resources\ScienceDefs.cfg ('UTF-8-SIG', 1.0)
Squad\Parts\Utility\commsAntennaDTS-M1\commsAntennaDTS-M1.cfg ('UTF-8-SIG', 1.0)
Squad\Parts\Utility\ladderTelescopic\ladderTelescopic.cfg
    [0:0]  (feff)
Squad\Parts\Utility\ladderRadial\ladderRadial.cfg
    [0:0]  (feff)
Squad\Parts\Command\advancedSasModuleLarge\advSasModuleLarge.cfg
    [261:41] â„¢ (2122)
Squad\Parts\Utility\ladderTelescopicBay\ladderTelescopicBay.cfg
    [0:0]  (feff)
Squad\Parts\Science\GooExperiment\gooExperiment.cfg
    [0:0]  (feff)
    [19:16] â„¢ (2122)
Squad\Resources\ScienceDefs.cfg
    [0:0]  (feff)
    [20:237] â„¢ (2122)
Squad\Parts\Utility\commsAntennaDTS-M1\commsAntennaDTS-M1.cfg
    [0:0]  (feff)

The number after the encoding is a certainty that ranges from 0 to 1. All the files are ASCII, except that six start with the byte‑order marker U+FEFF. In other words, they are encoded in utf-8-sig, which is idiotic. The detection code thinks one additional file is in Latin-2, but it’s wrong (well, it’s right, but it’s also UTF-8); the only non-ASCII character besides U+FEFF in these files is the trademark symbol, which is perfectly fine UTF-8.

TL;DR: When you open one of the six utf-8-sig files, you will probably need to strip the first character, or your parsing code will probably reject it. I just added the line str = strip(str, '\ufeff') just after reading the file into a string, which covers all cases. Once you do that, the rest of the file/string is UTF-8.

Edited December 19, 2014 by taio

[taio]

Let me know if you’d like to take a look at my code; I’ve got it working now. Julia is similar to Python. Though it only parses the nodes into an internal representation, and not a ConfigNode object, you might still find it helpful.