Discussion thread - BD Armory AI tournaments

[colmo]

 

 

 

Poll: Multiple choices allowed, check what matters to you. Note that Open classes means anything goes in matchups, different classes means like battles like.

Current AI v AI contests and tournaments:

Top Gun AI - Stock modern fighters - http://forum.kerbalspaceprogram.com/threads/130238-Top-Gun-AI-The-Official-Tournament-Thread

WW2 BAD-T - Early 20th century fighters, FAR and mods allowed - http://forum.kerbalspaceprogram.com/threads/130035-WW2-BAD-T-World-War-2-BDArmory-AI-Dogfight-Tournament-OPEN

FAR Fighter Challenge - A stock 1v1 contest using FAR and Dynamic Deflection mods. - http://forum.kerbalspaceprogram.com/threads/132274-FAR-Fighter-Challenge-BD-Armory-AI

[ASC] Air Superiority Challenge - King of the Hill - 2v2 AI Dogfight Battles- A modded 2v2 contest, no holds barred!

YouTube playlist of test videos, use #TopGunAI in title, description or tags and I can add your video to it:

I had no interest in BD Armoury until I saw the first AI dogfighting videos. That tipped me over the edge - I haven't built many planes since about 0.15, but I've been churning out fighters for the last few weeks.

The next step was to test them. Doing a human trial is far too subjective, so letting the AI do it seemed much fairer.

I quickly found 2 v 2 battles, where each team starts either side of the runway, all AI active and guard mode on, until I switch one to team B. At that point, all the planes scramble and I have plenty of time to switch the wingman to team B too. I then sit back and watch to see which plane wins the most fights, probably best of 3.

Winning means having at least one plane under controlled, powered flight when the other team doesn't (in some cases, they ran out of fuel!). If there's a mutual wipeout, it's a draw. I guess a whitewash, a win with both planes still flying, is worth a bonus point. Planes that fly out of guard mode range before they can turn and engage are deemed lost.

My question is, would anyone be interested in a builders' contest, designing planes and then having a knockout contest to find the winner, using this format?

Also, would there be interest in recording videos of the battles? I find them mesmerising! My PC, however, is very old, so I'm unsure how good performance would be.

Finally, would stock plus BD be the way to go, or allow a few select mods? Would different classes of plane be desired - e.g. twin turbofans are much more dangerous than single turbofans, and wouldn't be a fair fight. Perhaps old-school prop engine dogfights would be desirable. I'm unfamiliar with FAR, and would prefer to conduct the tests under stock aero,but as the AI does the flying and everyone else does the building, that may not matter. Weapon loadouts may also be open or restricted, again depending on feedback. AI settings would be whatever the plane's craft file is set to, though there would be max and min altitudes set.

I found that pure performance wasn't the only factor in design - redundancy and using tough parts play a part, in a speed v toughness tradeoff which challenges builders.

My house rule for part clipping is it's fine unless both parts are fuel tanks (I wouldn't count things like the engine precooler, as it has barely any), in which case only one can carry fuel, the other must be drained, or both only partially filled depending on degree of overlap.

In the course of this thread, I've developed a few more ideas:

colmo said:

- Arms Race AI: a no-holds barred, anything goes contest. Mods allowed (there are other BD based weapon mods, are they balanced?), all weapons open. Planes and anything else you can control using the AI. The only limit would be part count. The winner is the last one with something left still with propulsion and under control/controllable. For practical purposes, 1v1 is advised at the point of launch.

- Top Gun AI Pro: A FAR based version of the fighter challenge. A few select mods would be allowed (e.g. Pwings, Quiztech), the goal being to produce gorgeous and very effective planes. This is for the plane designers who want to take it to the next level.

- Joint Strike Fighter AI: A series of challenges to find the best allround airframe, based on the Top Gun ruleset. I envision three rounds:

*Air-to-air (v the same plane, probably a 3v2 scenario to push the designs to cope in an outnumbered scenario),

*Air-to-ground (testing their ability to take out ground defences and survive Goalkeepers and SAM missiles), and

*Interceptor (scramble and defend the KSC from incoming bombers, on a set flightpath targetting probes with weapon managers placed on the KSC, testing their ability to launch and attain altitude then take out large, turreted bombers).

Points would be awarded for each round depending on value and number of targets destroyed, losses and damage sustained. The only changes allowed between rounds would be AI settings, weapon loadout (hard points would need to be the same in all rounds, so give those careful thought), and fuel levels.

This one may take some improvements in the AI to pull off, Guard range being the biggest limitation at the moment.

- Duck Hunt AI: Use the Air Race features in Kerbal Konstructs to time a plane in taking out a series of targets, placed on the ground (or floating in the air) in series which the plane will follow in order. Mountains would make it extra challenging. I recall this being a challenge a very long time ago, using balloons. Now we can do it with AI.

- Top Gun VTOL AI: A variant of the fighter format, but especially for VTOLs. An arena lacking a runway would be perfect. I'll let you figure out how to do VTOL take-off using the AI. It can be done

I'd perhaps consider this and a few select mods, as stock currently has very restricted VTOL engine options: http://forum.kerbalspaceprogram.com/threads/107802-Throttle-Controlled-Avionics-2-3-0-1-0-4-Continued/page25

On part clipping, my thoughts are:

colmo said:

When I started this format, I thought it was intuitively obvious what parts were OK to clip, and what was not. Evidently not. If you want more engines, then you have to pay the design penalty of increased fuselage size and drag.

I'd suggest that parts you can't clip are engines, cockpits (or at least their glass canopy), or fuel tanks with other fuel tanks.

Parts that make sense to clip - structural, intakes, fuel tanks (as long as not with other fuel tanks, unless one is drained of fuel).

For guns, as long as the muzzle is not clipped, and missiles you obviously don't want to clip!

Edited April 18, 2016 by colmo
Added links to tournament threads

[Lofar968]

The 2 v 2 format is pretty standard I think. Also base assaults might be cool to do. I would be interested in partaking in this.

[colmo]

The 2 v 2 format is pretty standard I think. Also base assaults might be cool to do. I would be interested in partaking in this.

Thanks! I'm thinking of minimal human input for reproducibility - any ideas how to set up base assaults so the starting conditions are always the same without a massive time investment? I might use some persistence file editing to set e.g. starting positions, because I noticed slight variations in heading tend to result in the same thing happening over and over, such as one plane getting targeted by all the missiles from both enemies.

I do like including take-off as part of the test, because the planes to reach min altitude first have the advantage. I even figured out how to do VTOL take-offs under AI, though sadly the basic jet has been nerfed so badly it's not worth it.

[HoloYolo]

Stock and BD. Trying to keep as stock as possible.

[Darren9]

One stock and one not class seems best (as long as enough entries), maybe a part limit as 4x planes can add up and someone has to film it.

[Noname117]

If you get enough people interested in this we could set it up like a tournament: Each player gets to have a team of 2 aircraft (These can be the same aircraft or 2 different ones) with certain restrictions (likely including weight, part count, and armament) and is pitted up against an enemy player's team. The battle is entirely AI fought. Whoever wins, say, best out of 5 rounds moves on to face another winning team while the losers, well, lose. Last team left at the end wins. The problem with this is you need a lot of people participating (at least 8 probably for good fun, though 16 would be recommended), along with at least a couple of people hosting and recording the challenge (Though it could be a good livestream event).

If anyone else who livestreams is interested, do you want to get this set up? One of us could livestream one half of the battles, the other one the other half. PM me if interested

Edited July 19, 2015 by Noname117

[colmo]

If you get enough people interested in this we could set it up like a tournament: Each player gets to have a team of 2 aircraft (These can be the same aircraft or 2 different ones) with certain restrictions (likely including weight, part count, and armament) and is pitted up against an enemy player's team. The battle is entirely AI fought. Whoever wins, say, best out of 5 rounds moves on to face another winning team while the losers, well, lose. Last team left at the end wins. The problem with this is you need a lot of people participating (at least 8 probably for good fun, though 16 would be recommended), along with at least a couple of people hosting and recording the challenge (Though it could be a good livestream event).

If anyone else who livestreams is interested, do you want to get this set up? One of us could livestream one half of the battles, the other one the other half. PM me if interested

Well, we can have few builders but many planes - I have a load of differing designs I'd happily submit to make up numbers. I don't have experience of livestreaming or YouTube, so happy to let someone else do that (there's no requirement all the contests be performed by the same person if the starting conditions are meticulously agreed).

I'd agree on part count - no 100 part monstrosities featuring all manner of weird parts just to make it look like an F22, this is about effective designs, and if they're original or only work because it's KSP, so be it, all to the good.

[Noname117]

I feel like we should try to reduce the number of tiny entries as well, we should have some sort of system encouraging effective but somewhat realistic designs. I'm thinking we do that by having a minimum on the amount of fuel a plane can carry (say, 600 units of fuel for each turbo, 300 for each basic jet), and a maximum part count and weight limit (IDK about the weight limit yet, I'm thinking the part count limit should be at either 75 or 100 parts). Weapons should also be limited (I'm thinking a points system, where each allowed weapon and/or ammo box is worth a certain amount of points, and there is a max points limit).

I feel like the craft should be large enough to be unique yet small enough to keep the game playable.

[colmo]

I feel like we should try to reduce the number of tiny entries as well, we should have some sort of system encouraging effective but somewhat realistic designs. I'm thinking we do that by having a minimum on the amount of fuel a plane can carry (say, 600 units of fuel for each turbo, 300 for each basic jet), and a maximum part count and weight limit (IDK about the weight limit yet, I'm thinking the part count limit should be at either 75 or 100 parts). Weapons should also be limited (I'm thinking a points system, where each allowed weapon and/or ammo box is worth a certain amount of points, and there is a max points limit).

Remember that running out of fuel is effectively a loss, and running out of ammo is a death sentence. I haven't really tried to abuse the new aero to make a tiny plane (which I presume would have monstrous TWR and low drag), but it's likely it would be limited in it's endurance, toughness (mk1 fuel tanks go poof very easily, mk2 are much sturdier) or weapons carrying capability. I'd be interested to see if an abusive design could make it all the way through before getting exposed by one of its design compromises? It's possible this ruleset could evolve over multiple contests.

- - - Updated - - -

That being said, a minimum delta V (from Kerbal Engineer readout) would be an easy restriction.

[Noname117]

That requires having KER installed. I'd say it would just be easier to limit the total amount of fuel an aircraft can have based on its engines. Speed and maneuverability are going to be major advantages in the competition, and tiny aircraft would be easy abusers of that, along with being harder to hit. I feel like the fights of the competition should mostly be between aircraft which look vaguely like fighters rather than squished (horizontally) cartoony aircraft.

[colmo]

That requires having KER installed. I'd say it would just be easier to limit the total amount of fuel an aircraft can have based on its engines. Speed and maneuverability are going to be major advantages in the competition, and tiny aircraft would be easy abusers of that, along with being harder to hit. I feel like the fights of the competition should mostly be between aircraft which look vaguely like fighters rather than squished (horizontally) cartoony aircraft.

It does have simplicity going for it. I'm a tiny bit curious to see if someone goes the other way and builds a flying fortress replete with Goalkeepers, but first time out, reasonably similar planes avoids too much silliness. We can always expand the parameters for subsequent contests.

- - - Updated - - -

I just added explanation of my classes options. Noname is advocating a fighter class as the main contest, which fits the different classes option rather than open classes.

[Lofar968]

Thanks! I'm thinking of minimal human input for reproducibility - any ideas how to set up base assaults so the starting conditions are always the same without a massive time investment? I might use some persistence file editing to set e.g. starting positions, because I noticed slight variations in heading tend to result in the same thing happening over and over, such as one plane getting targeted by all the missiles from both enemies.

I do like including take-off as part of the test, because the planes to reach min altitude first have the advantage. I even figured out how to do VTOL take-offs under AI, though sadly the basic jet has been nerfed so badly it's not worth it.

I would say using Kerbin Side for placement then you could probably just duplicate the save file. I agree with including take off as the shorter the take off time the less fuel used for it and the quicker you can get into battle.

[colmo]

I would say using Kerbin Side for placement then you could probably just duplicate the save file. I agree with including take off as the shorter the take off time the less fuel used for it and the quicker you can get into battle.

I suppose we could create a custom Kerbal Constructs site with 4 launch sites, then its just a matter of launching them, setting them up, and letting them go for the 2 v 2.

As for base assaults, similar, but maybe with a hill in the way to give the planes a chance to get in the air before getting shot at.

If this develops a following, new contests would use new custom sites which would be akin to a golf tour with unique courses

Edited July 20, 2015 by colmo

[Lofar968]

I suppose we could create a custom Kerbal Constructs site with 4 launch sites, then its just a matter of launching them, setting them up, and letting them go for the 2 v 2.

As for base assaults, similar, but maybe with a hill in the way to give the planes a chance to get in the air before getting shot at.

If this develops a following, new contests would use new custom sites which would be akin to a golf tour with unique courses

Kerbal Konstructs? (Went and looked it up...) Ah yes we could do that. Well I thought you'd put the base far out as in on the Island Runway or other places. You could also do a naval assault. (Although that would be heavy on part count.)

[colmo]

For weapon loadouts, it's really a matter of Sidewinders, AMRAAM, and guns (fighters would surely all use the hidden Vulcan).

I found myself restricting my single engine class jets to two Sidewinders and one Vulcan, and twin engine to two Sidewinders, 2 AMRAAM and 2 Vulcan. Ammo was one box per gun. They also got more countermeasures, 4 instead of 2. They rarely got shot down with missiles.

The points system idea means some may go for guns, some for missiles, some for a balanced mix. As it's really a test of the plane, not the weapons minmax, I favour a standard loadout for particular classes of plane.

What's the usual loadout of an air superiority jet?

[Noname117]

The problem with trying to make this into several competitions all with their own variations on the rules is that you need people to participate in them all, and I don't think we have enough participants within this community at the moment. We have to restrict it down to a manageable number, probably with there being only one competition at a time. This is why I am suggesting a somewhat restricted fighter class, since that would be the design most would choose for a competition like this. I feel like we should use that as our building block and try to prevent designs from deviating too far from those realms to keep the competition fair, while trying to give enough leeway to make the designs have some variety. It's a thin line to tread, and I'm just thinking of the best ways to do it.

If need-be I could host this competition (likely with someone else who streams on twitch) and have it be seperate from yours, so we don't have to argue much.

[colmo]

No need, as you say, no point splintering the participation. I'm happy to put the idea out there and let someone else lead with it. Your enthusiasm is noted

Key is that people design for challenges rather than use existing craft (they may modify), so you may be on the money by keeping it focused.

I've just been reading up on interceptors, which throws up a whole different set of scenarios.

The ultimate is a triathlon of air superiority, interceptor and ground attack to find a multirole design.

Weapons loadout changes with role, so I think having standard weapon loadouts for particular roles is a good idea. I note some real fighters have one cannon, some have two - not sure whether to restrict or allow swapping one for, say, 2 Sidewinders?

[Noname117]

I'd say just adopt the weapons point system. Make AMRAAMs fairly expensive, sidewinders a bit cheaper, each cannon comes with a small cost with a heavier cost per ammo box. I'm thinking the hidden 20mm vulcan should be allowed, along with all spinning non-turret guns (Though including the .50s), though their firing arcs should be restricted fully (so the essentially become forward firing weapons). The larger turret weapons (The goalkeeper, the M1 turret and the millennium turret) shouldn't be allowed; same for the howitzer. Air-to-surface missiles and surface-to-air missiles could be disallowed, but I do know that all of those can shoot down aircraft.

Standard weapon loadouts for each role is likely not a good idea. Different planes have different weapons capabilities based on role and cost, and ultimately different individual missions would have planes with different loadouts. I think placing restrictions on how much a player can mount on an aircraft is better than assigning said aircraft certain weapon layouts.

Also, maybe there should be a rule that you can only mount missiles on hard-points, to encourage more realistic designs and add just a bit more of an engineering challenge. Of course, hard-points don't have to be BD-s hard points, they could be other pieces used to mount the missiles lower than surfaces it would clip into.

Edited July 20, 2015 by Noname117

[Lofar968]

I'd say just adopt the weapons point system. Make AMRAAMs fairly expensive, sidewinders a bit cheaper, each cannon comes with a small cost with a heavier cost per ammo box.

So give the builder a set amount of points, then make all weapons and ammo (allowed) cost a said amount of points? The sounds reasonable to me.

[Noname117]

Yep. Make AtA missiles cost a fair number of points, make cannons cost a smaller number of points but make ammo for the cannons cost more. ATG missiles and cruise missiles should cost less than AtA missiles, as they would be less likely to hit targets (a cheaper missile solution, could be used effectively by some, allows more creativity in weapon designs). Rockets should cost a fair amount, as I assume a hit with them would be devastating. Damage, fire rate, range, and hit chances should all be taken into account when assigning a weapon points, as a 30mm cannon should be more expensive than a 20mm.

Also, parts of this would likely require testing with the AI, to make sure they will target air targets with ground attack based weapons.

Well, I'm off to bed. I'll be back tomorrow.

[Bob_Saget54]

If this goes through, count me in. I got a fighter design I'm itching to test out

[colmo]

I'm getting the feeling the testing for this challenge might be as much or more fun/work as the tournament...

Easy way to test weapon costing is to test different loadouts on the same airframe against each other. Standard seems to be 2 Sidewinders, 4 AMRAAM and a cannon, but I doubt there'd be 4 AMRAAM still on planes in a close dogfight like this. I'm not sure which missile type is better inside 5km, surely the Sidewinder's lower mass would make it more valuable?

Countermeasures also need including in the cost, to introduce a defence/offence dynamic.

[Lofar968]

I'm getting the feeling the testing for this challenge might be as much or more fun/work as the tournament...

Easy way to test weapon costing is to test different loadouts on the same airframe against each other. Standard seems to be 2 Sidewinders, 4 AMRAAM and a cannon, but I doubt there'd be 4 AMRAAM still on planes in a close dogfight like this. I'm not sure which missile type is better inside 5km, surely the Sidewinder's lower mass would make it more valuable?

Countermeasures also need including in the cost, to introduce a defence/offence dynamic.

Well the AMRAAMs are bulkier in close range but also do more damage than sidewinders. I can't say which is better. More maneuverable sidewinder or the more damaging AMRAAM?

We should decide on how many 'points' we get before we move ahead of our selves.

[colmo]

Part of the builder's skill will be tweaking the BD AI to optimise the performance of the craft. Steering factor and limiter etc. are to be used exactly as provided in the craft file.

Should max and min altitudes be the same for all? Some designs may work better high, other lower and slower, so may also be a design choice? Or should this, too, be a design choice?

As long as the ranges overlap reasonably, this shouldn't result in the silly scenario of two sets of planes circling, one so far above the other they go out of the 5km engagement range.

[Lofar968]

Part of the builder's skill will be tweaking the BD AI to optimise the performance of the craft. Steering factor and limiter etc. are to be used exactly as provided in the craft file.

Should max and min altitudes be the same for all? Some designs may work better high, other lower and slower, so may also be a design choice? Or should this, too, be a design choice?

As long as the ranges overlap reasonably, this shouldn't result in the silly scenario of two sets of planes circling, one so far above the other they go out of the 5km engagement range.

I can agree with your first statement. As for altitudes, generally being higher than your opponent can be a good thing. As diving is a lot easier than trying to climb to kill your opponent, while climbing isn't easy it's still possible. I would say decide a min/max altitude range for the select tournament and go with that.