Suggestion: Adjusting Mansion Lootpool Output (PA!) Sept 16, 2020 14:59:29 GMT -6
Post by Darky on Sept 16, 2020 14:59:29 GMT -6
It’s time for the No Good Bad Math Statistics and the Devil’s Advocate Job of the Century.
As discussed in the Discord channel, I’m making a formal request to have the Mansion’s current system tweaked towards decreasing the output of it’s rewards. The manifesto for such includes the following points:
- The rewards are Too Damn High, statistically speaking.
- Economically speaking, outperforming and overshadowing every other loot area on the site, at nearly 2:1.
- Doesn’t require as much investment as other areas in terms of equipment, skills and narrative restraints.
- The Mansion’s original design and subsequent adjustments prior to May 2019 weren’t intended for this kind of output.
- With the recent change to ruling
which allows training in loot areas, combined with the Mansion’s loose
regulation on narrative, the potential for abuse is high.
With that presented, I’m well aware this is not exactly going to be a popular choice, and several people had already made their personal counter-points to the idea based on several other factors. This is why I’ve compiled a comprehensive statistical analysis of the Mansion and it’s output during the course of the entirety of the HxR Freeforums era. I will attempt to address as many of these concerns in this evaluation as possible.
Disclaimers: For the purposes of avoiding mixing results from different case handling methods, threads where 2 or more characters roleplayed together were not included in this statistic. A general overview of skills was accounted for, but will not be included in the final results as it’s unclear which specific skills work which way. Skill affect is only noted in observation, based on patterns, but without detailed correlation. Additionally, pet contribution was not counted in the statistic for the same reason, as it’s unclear which aspects or types of pets, if any, contribute to results - observable results were inconclusive. Posting prizes were not included in this statistic.
Now it’s time for the Deep Math.
TL;DR - Things be broke, need tweaking, here’s why.
The Yeet Curve, or How Output Suddenly Changed
Prior to a certain point in time, the Mansion’s output was generally quite low, due to the fact that it was built as a free-form roleplaying area where one can also find loot, succeeding Spirit’s original test with the VMM Graveyard event. It was because of this principle of being free-form heavy with no specific objective or narrative (only a change in backdrop themes) that weighted the loot drop chance towards the rare - a player was able to put in minimal effort, but had to invest more time as a result. The Wishing Well was a later addition intended to help offset the low drop rate of items with a semi-guaranteed choice of special pre-determined rewards.
While the output of loot can always vary from person to person, thread to thread and even time to time, the overall summary of the Mansion shows a distinct and sudden change that can be statistically tracked, as follows.
Here’s a table listing the highest item drop per 5 posts for each month:
If that seems a little plain and unassuming, here’s the same table as a line graph, to put it into perspective:
I also included the lowest item drop rate per 5 posts (blue), to demonstrate the baseline in comparison.
Between April 2019 and May 2019, something was changed in the system as to alter it so drastically, the output curve basically quadrupled by June 2019. The initial influx could have been a behind the scenes test, since the addition of a new skill (Architect) - that has since been hinted to be linked to the Mansion - was announced later in August 2019.
From that point onward, in nearly all the sample cases, as well as the overall graph itself, the output has been steadily climbing, while the lowest drop rate has (almost) remained the same. This means the system wasn’t adjusted overall, as the baseline hasn’t changed - the low rate is still the same. Instead, the statistics suggest that only the reward ceiling was adjusted, to allow more rewards per roll than previous.
But this wasn’t happening in all scenarios. Even in some recent cases, the samples continued to have the same low reward rates as before the adjustments. This can thus be correlated to some player’s obtaining relevant skills and thus being able to increase their output drastically compared to players who didn’t, thus preserving the low rate curve (almost) unchanged.
It should be noted that the spikes of November 2019 and June 2020 in the baseline are largely due to a low sample amount for those months largely consisting of high-skilled players, thus they are slightly anomalous as their counterparts did not produce comparative results.
February 2018 is anomalous for the same reason, as there wasn’t a big enough sample size for comparative results.
TL;DR - Something was tested, then changed, and the output got yeeted times four. Not everyone benefited from this, tho.
Just really good luck? Not quite so simple..
“Once is accidental chance. Twice is good luck. Three times is a statistic.” - While all the numbers presented here can sound definitive and be argued this way and that to infinity on what’s a ‘good’ number, keep in mind that because the system is based on chance and rolling, these are just the results that we can see - the successful results. This means the actual dice at work behind the scenes can be even greater. While we can’t ever know for sure how many dice are truly at work, we can compile statistics based on visible results and formulate re-occuring patterns based on that. The more a pattern occurs, the more we can see how things actually work in practise.
So far, these patterns let us deduce one of two things: 1) the prevalence of successful rolls means the likelihood of success per roll is high in skilled sample cases; or 2) there’s several more rolls that could be happening in addition to the results we see.
In the first case we assume there’s a lower or limited number of rolls and they’re swayed heavily in favour of the player, which is a case that can be made when looking at sample sizes from several posts rolled for at a time (10-20 posts, for example) and seeing an average result of 7-10 items from such rolls. It can then be argued that it’s just equivalent to the higher number of posts, meaning the more posts you make, the better your results (“Hard work brings results”).
However, there are a few select cases that show just 5 posts alone have produced results of 5-6 items for those rolls (And that’s not counting posting prizes), which indicates 2) is also true:
(Posts themselves blurred for anonymity)
They show there’s a lot more rolls happening per calculation, even when the sample size is the smallest it can be (5 posts, or 1 roll event), which means the results of 7-10 from those 10-20 posts might actually be low or generally considered below average, in that case. The pattern there is, if 5 posts can produce 5-6 items, 10-20 posts have the chance to produce at least 10-24 items. That’s essentially 1-1.2 items per a single post. No other loot area does this.
TL;DR - Chance and luck in great enough sample sizes can be quantifiable. By increasing both chance and sample size regardless of effort, the results are no longer random and success is more a guarantee than luck. Thus practical results can be inadvertently pushed beyond reasonable capacity.
But free loot is good, right? Well, Yes and No..
No-one hates free money or free loot. In fact, it’s very difficult to say no to it, when presented, especially when it’s in a great enough quantity to enact big changes. Loot has a worth which can be translated into credits, and credits can be used to solve most character-related stepping stones towards character progress, be it purchasing items from official shops, creatures from other players or resources needed to obtain specific items (Such as nodes).
For Mansion items, the average item has a worth of 25 credits, collectables are worth 10 credits, and posting items can range from 10-50 credits depending on item. Pets and Remnant crystals have worth that is less clearly defined, but given the demand on nodes, the latter can at times match their magical cousins from the Mines.
Items also have a value, however. "Worth" is quantifiable in numbers, but "value" is dependant on the item’s role in the ecosystem that it inhabits. Prior to the changes, the value of Mansion items was attributed higher due to their inherent rarity and the difficulty it takes to obtain them. This caused them to have a higher worth.
With the changes to their rarity, their value has been intrinsically changed to be cheaper than before, as they are much easier to obtain now, and in much greater numbers than before. But the changes never went past adjusting their value as their worth remained the same - the regular items are still worth 25 credits, despite the fact that you could potentially obtain 4-5 of these items in 5 posts, where it used to take someone 20+ posts to do so. Compared to the Mines, where 5 posts might get you 1-2 items generally ranging in 5-20 credits in worth.
What this means is the economy of Mansion items is off.
If the goal of a player is to obtain a lot of items for their collection, they are not concerned with worth, only that the value is favourable. A player looking to make bank is concerned with both value and worth, which makes the Mansion a decidedly better place to grind than the Mines, Garden or Fishing, since not only is the Value-to-Worth ratio much more stringent there, they do not need tools, many skills nor adhere to any specific roleplay constraints to do so.
Essentially, it damages the other areas in terms of their economy. Not only would a player not necessarily need to roleplay there to obtain results, they can use Curious Collections to bank their Mansion items for even greater worth from those areas.
For example: Right now, Crystal #3 is available in unlimited quantity in Curious Collections. It can be traded for 30 worth of Any item, so Mansion finds can be used to trade for a multitude of Crystal #3’s. The Crystals (regardless of #) sold to The Underground have a worth of 30+1d6, however (Resulting between 31-36 credits per crystal). This means the player can bank even more worth in credits out of this method than they would when selling Mansion items normally. Because this is so much more lucrative, there’s less incentive to participate in Mining, especially considering the Mines’ rewards have a fluctuating worth based on rarity, while the Mansion’s is always a guaranteed worth across the board, and there's a guaranteed item in Trader each month that Mansion items can be swapped for.
The Value-to-Worth of the Mines is outshadowed by the Value-to-Worth of the Mansion, and open to severe abuse.
TL;DR - Drop chance was changed, but the price remained the same, meaning the system it open to abuse. Economy no do good.
..and Why It’s Complicated
On the surface the idea of more money doesn’t seem inherently bad, but it can affect the long-term experience of the game. Essentially, when someone has achieved all there is to achieve, obtained all there is to obtain, the incentive to keep going diminishes. It’s a basic game design principle that causes open-world games to sustain themselves post-death purely with new mods, and why MMO games run a constant marathon of updates in order to stay just a little bit ahead of the death curve.
Instant gratification leads to instant actualization. In game design, the balance between instant and delayed gratification is the most delicate part of the game, as you don’t want to have it be over before the player has even begun - Make progress too quick and there’s no challenge, causing boredom. Make it too slow and it’s grindy, causing frustration. Either side of the spectrum threatens to lose the player's interest in the game, even if on just on a specific level.
This is why the economy of any game, especially loot-based RPG games, is so important. It can make or break not just the system it affects, but adjacent system by proxy, as each system is connected to every other in a surprisingly delicate ecosystem.
Quick rewards can feel nice when getting them, but if a collector is able to complete their Mansion collection within a year of starting (Obtaining all items for every month, as each month progresses), they have no objective once the months begin to repeat. The game isn’t designed to stop them from completing their collection - the task is not intended to be impossible - but if they only need to participate in each month once, there’s no longevity to the design past that one month, either. Past that point, the collector has no incentive to participate and no long-term goal to achieve, because it was completed short-term.
The Wishing Well is a prime example of this. It takes just 12 collectibles (2 + 4 + 6) to obtain every item in the Wishing Well for that month, but there’s sample cases where players have had some 20-30+ collectibles still left over. As Wishing Well items repeat year after year, once a player obtains all of them once, they have no need for collectibles after that, much less in that sheer amount.
A player interested in making bank can continue past that one month to obtain a fortune in quick succession, but their progress and goals end once they’ve succeeded in purchasing everything they were looking to obtain with the payment.
For example: One of the longer sample cases had a net worth profit of 9137 credits for all the posts done in 2020 alone. To buy every single item in Supply and Demand once (not counting recolours) would cost 6185 credits. They could buy out Supply and Demand, and they would still have 2952 credits left. That player would never need to visit Supply and Demand again (unless new items were added).
Instant actualization can be alleviated through roleplay, but only up to a point. A character who owns everything can still have engaging plot, but it requires cooperation from other players (in many cases). Forging narratives to include a journey can prove to be challenging, when there is no challenge left for the character to face, however. There has been a tendency for players to show less interest in activities that do not include a reward, than activities that provide tangible results as well as fictional. In competitive scenes, other players might not even wish to engage with players who are clearly above them in equipment or preparation, regardless of their own progress.
TL;DR - Game design is hard. It has many moving parts that are connected - changing one thing can change many others. So if one thing breaks, it can break others, even if the consequences might not be immediately evident.
Right, so everything sucks and it needs to be all torn down, is that it?
Not at all. As mentioned here and in Discord, tweaks can be made big and small. Mistakes can also be made big and small, but the important part isn’t who made it or why - the important part is what is done after it is detected.
Simply put, something was changed and it is working a little TOO well, but it can be adjusted to be more manageable and provide a solution that can satisfy all parties involved. A system’s behaviour can be observed and it’s effectiveness calculated from a statistical standpoint, to inform how best to tweak it in order to be in better balance to the rest of the site. The first tweak might not do it. Even the fifth tweak might not do it, but it’s important to keep observing and trying, until that magical balance recipe can be obtained.
Balance? In this game? It’s More Likely Than You Think
Here-in I can only make assumptions based on the statistical observation of the system so far and suggest options which can be used to fix the issue at hand, while not robbing it of the overall goal to improve the Mansion experience.
As shown in the data above and the Deep Math sheet, by and large the output of the Mansion hovers somewhere around 3-5 items per 5 posts, which is too much. However an average of 1-2 items, more similar to other areas, can satisfy a sense of accomplishment while also toning down the output’s severity, so it’s more manageable.
The caveat here is that I do not know the system’s specifics, but I can make recommendations based on these observations, as follows:
- Adjusting Value: If the overall goal
is to make things easier to find, the skills should affect either the
find chance or dice amount, but not both. The increase of both caused a
multiplied effect that did nothing to adjust the actual baseline.
- Adjusting Worth: If Mansion items are
to be more common, their worth should be adjusted accordingly, so it
doesn’t overshadow other areas, especially through the use of the
Trader. Higher find chance should mean cheaper prices.
- Adjusting Collectibles: In many sample
cases the collectibles matched regular item finds nearly 1:1,
suggesting a separate existence from regular loot pools. If collectibles
have their own chance and dice amount, it should be reduced
accordingly, perhaps even regardless of anyone’s skill investment, due
to their very limited time of having value.
What the specific values of these adjustments are? No clue. It would require someone to run math from the mod side to see what the relationship between loot baseline and ceiling is in detail, since they will be able to see specific numbers of rolls and chances. From my side, I can only estimate based on observable results and theory from reoccurring patterns, but I do not have details to give. That part would be up to mods.
TL;DR - System big, mysterious and scary, but tweak suggestions, here’s a few.
And that's basically my TED talk, danku.