- A Brief Overview
- Problem 1 - Bad Naming
- Problem 2 - The same fields can be/is used for completely different things
- Problem 3 - Not every field is used
- Problem 4 - Not every row is used
- Problem 5 - Nothing guarentees a field is used correctly
- Using Datatables Correctly
Datatables have a multitude of issues, making them completely unsuitable for data mining, reverse engineering and modding. Rather than using datatables, you should always go back to the source, and extract values (or mod them to new ones) from there.
As the name suggests, datatables are tables of arbitrary data.
Every datatable defines it's own custom row struct. For example, the datatable:
/Game/GameData/Balance/ChallengeRewards/Table_ChallengeRewards_CrewChallenge.Table_ChallengeRewards_CrewChallenge
Defines the row struct:
struct RowStruct {
float Experience;
float Cash;
float Eridium;
};Note that the field names commonly end with a hash, which has been deliberately ommited.
The data table itself then consists of a mapping of names to row structs. In this example:
| Key | Experience | Cash | Eridium |
|---|---|---|---|
Salvage |
0.08 | 1.0 | 10.0 |
Sabotage |
0.08 | 1.0 | 10.0 |
TyphonJournal |
0.08 | 1.0 | 15.0 |
DeadDrop |
0.08 | 1.0 | 5.0 |
Kill |
0.08 | 1.0 | 10.0 |
Hunt |
0.08 | 1.0 | 15.0 |
HiJack |
0.08 | 1.0 | 20.0 |
EridianWriting |
0.0 | 0.0 | 25.0 |
So now how is this data actually used? There are two custom structs which point into a datatable.
struct FDataTableRowHandle {
class UDataTable* DataTable;
struct FName RowName;
};
struct FDataTableValueHandle {
class UDataTable* DataTable;
struct FName RowName;
struct FName ValueName;
};Then there are engine functions which will use these structs to extract the relevant value, for use in further processing.
The general FAttributeInitializationData struct used all over the game pretty much whenever there's any numeric value contains a FDataTableValueHandle field DataTableValue. This means, generally speaking, any number in the game can be sourced from a datatable.
Now datatables don't need to only include numbers. It is relatively common to find string comment fields for example, but there's nothing stopping them containing more complex nested structs or arrays. These generally have more hardcoded usages, so won't be able to be replaced by the techniques discussed later.
A big problem with datatables is that the extra level of abstraction makes naming a lot more important, and makes working with bad naming a lot more annoying.
The example I gave above is one of the few good datatables in the game. It is not uncomon to find tables like the following:
/Game/Gear/Weapons/_Shared/_Design/GameplayAttributes/_Unique/DataTable_WeaponBalance_Unique_ATL.DataTable_WeaponBalance_Unique_ATL
struct RowStruct {
float DamageScale;
int MinGameStage;
int MaxGameStage;
struct FText Custom_A_Description;
float Custom_A;
struct FText Custom_B_Description;
float Custom_B;
}What do Custom_A and Custom_B mean? Or what about:
/Game/PatchDLC/BloodyHarvest/GameData/Balance/BloodyHarvest/DataTable_Season_Halloween.DataTable_Season_Halloween
struct RowStruct {
float DamageScalar;
float Cooldown;
float Radius;
float Secondary;
float Tertiary;
FString Comments;
}Again, what do Secondary and Tertiary mean? The only way to tell is to go find where they're referenced.
This problem leads directly on from the last. A lot of the time, the reason they use these super generic field names is because the exact same field is used for a whole variety of things. This isn't limited to just those genericly named fields however.
Let's take a look at:
/Game/PatchDLC/Ixora/GameData/Balance/Table_GearUp_LegendaryLootOdds_Ixora.Table_GearUp_LegendaryLootOdds_Ixora
struct RowStruct {
struct FText Type;
struct FText Map;
struct FText Encounter_Type;
float LegendaryDropChance_Playthrough1;
float LegendaryDropChance_Mayhem;
float LegendaryDropChance_Playthrough2;
struct FText Comment;
};Firstly we have a classic naming problem - what's the difference between the three legendary drop chance fields? But let's look at LegendaryDropChance_Playthrough1. After peering through references, you'll find it's used in at least 5 different ways.
| Key | LegendaryDropChance Usage |
|---|---|
Enemy |
ItemPoolList.ItemPools.PoolProbability |
WhiteChest |
ItemPool.BalancedItems.Weight, multiplied by Arms Race mayhem multiplier |
World-Vending |
ItemPool.BalancedItems.Weight, multiplied by 100 |
RareSpawn |
ItemPool.BalancedItems.Weight |
FinalBossBonus |
ItemPoolList.ItemPools.PoolProbability, used twice in seperate rolls |
This means if you were to edit this field in every single row, say just adding 1, it will have 5 completely different effects.
And this is just looking how it sets the value of a single field elsewhere. If you include the different ways itempools/weightings are arranged (and thus the actual drop chances), pretty much every single row is used in a completely different way.
This problem leads directly on from the last example. We only looked at LegendaryDropChance_Playthrough1. What are the mayhem and playthrough 2 versions used for? Nothing. The fields have values (all the legendary drop chance fields use the exact same value in fact), so you might be led into thinking they have an effect, and say try to up all the mayhem ones, but it will do absolutely nothing.
Once again, we'll use the Arms Race example. There is a row for every chest room, so you might say try increase the rates of just the Dam row. It won't do anything. The actual drop chest room drop rates are all based off of the attribute:
/Game/PatchDLC/Ixora/GameData/Loot/ItemPools/Attributes/Att_GearUp_LegendaryWeight_EventChest.Att_GearUp_LegendaryWeight_EventChest
Where does this attribute pull from? Exclusively the CreatureSewers row. Every single chest room's drop rate is based off of the sewers's row, all the other rows are completely unused.
This is kind of the culmination of all the other problems. You can't tell what field a datatable value sets, or that it always sets the same field, without looking at every single datatable reference. You can't tell if a field or row is used without looking at every single datatable reference. And you can't tell if all the objects you expect to reference the datatable actually do without looking up every single one of them to check. There just is no way to make sure a field is being used like you expect it to. So if you need to lookup every other object anyway, what's the datatable actually doing for you?
Now datatables do exist, you can't just ignore them completely. How do you use them correctly? By getting rid of them as soon as possible.
When datamining, go directly to the source. Want to find the stats of an item part? Look up that part directly. Or for something a bit more complex, want to find the drop rate of a weapon? Lookup that item's balance, then lookup it's references for it's itempools, and the references to the itempools for higher pools, until eventually you find the root bpchar/itempoolexpansion/spawnoptions object.
Once you've found the source object, you can read the value you want directly off of it. A lot of the time you'll probably find there's actaully no datatable involved. If, and only if, you find it uses a DataTableValue field, lookup the row and value in the datatable, and then immediately close it and pay it no more mind.
When you get to modding the value, edit the full FAttributeInitializationData struct, delete the datatable reference, and set the constant to the same value. Don't just change the scale constant. And don't try edit the datatable. There is absolutely no difference between a DataTableValue and a BaseValueConstant which return the same value. Setting the constant directly avoids all the issues the extra layer of indirection adds, it makes your intent far clearer, and it avoids issues if another mod/a gbx hotfix edit any of the other fields along the way.