I think it is worth mentioning that if we do ban certain things (like data anaylzing posts using a program) that we then assume that players aren’t going to use those programs. They still could obviously, but that also goes for every rule on the site. We trust that players arent going to DM the other people outside the game and ask “is this post scummy or towny”, we trust that players wont share accounts, we trust players wont try to like hack into eachothers accounts. If we ban that, we also trust that players will not use those data anaylsis options so yes while they technically could just still use them that goes for literally every rule so i dont think its a concern
4 Likes
Dont allow any data stuff unless theyre for memes like zugs tictactoe one
1 Like
wait I’ve suddenly changed my mind
we should absolutely allow private, non-disclosed use of programs
because then I can develop and sell them
business opportunity 
7 Likes
If every post in the cookie threads had been automated in a forum game under this precept, this modreveal statistically would’ve happened ten times already.
1 Like
horse story
2 Likes
I know you guys are joking, but regardless of what we decide here, it will not be allowed to have a computer program break the rules (such as by posting your rolecard)
2 Likes
if a program is publically released and an error is discovered in it what would be the course of action taken by the staff?
can you be more specific, I think it would depend on the circumstances
yeah i figured so ill come up with some scenarios when i get home
my post was longer and cooler (i did not see yours)
2 Likes
a horse is also longer and cooler
1 Like
If were cracking down on AI, based on my FM games im pretty sure like 30% of players here are NPC bots so thats a good starting place to look into
4 Likes
I can confirm when we ran post delay analysis I was, strangely enough, reverse post delay polarised… most of my wolfgames had a faster minimum delay between posts, implying I was more comfortable in the thread as wolf. Weird, right?
Then I checked the specific wolfgames we were looking at. 2 mashes and a high-post Sorc17er. Of course I was posting faster as a wolf. And I also tended to post slower in my earlier games, and I randed town more early on. When you looked at comparable towngames and wolfgames, there wasn’t nearly as much difference.
When we were actually running this sort of analysis, the only actually workable correlations we found were the ones that were large enough to notice by human eye without it. Beancat was the example - she was more consistently active as town, with larger delays across the board as wolf. You could also just tell this by… looking at her postcount… or glancing at the timestamps of her posts… the difference was big enough that it didn’t matter.
We ran similar analysis on Achromatic and Kiiruma and found absolutely nothing. This is despite the fact that Kiiruma is was generally known at the time to be somewhat postcount polarised! The thing people noticed With Their Eyes - that he liked playing wolf more, so he posted more as wolf - didn’t show up on our spaghetti graphs, probably because what constitutes “posting more” is so contextual.
My reaction to actually Sneakily Trying Out this kind of analysis, in the most extreme of situations - nobody else had access to it, even knew we had it, they couldn’t counterplay at all - was “how the fuck did this ever become enough of a problem to get banned anywhere, it feels like reading tea leaves”.
I really do not think mafia threatens to become a programming competition so long as fairly reasonable guidelines are in place. Word choice analysis might be different, I dunno. But if it is, we can just ban that. I think the use-cases for mass data processing are pretty niche and certainly don’t trivialise the game.
3 Likes
@Lumi I was not intending to make an argument for the wholesale banning of statistics, but I can see how what I said could have been interpreted that way upon re-reading. On MU, there’s a page (that’s incredibly hard to find IIRC) that will show you all sorts of statistics about your play. Your vote accuracy as various alignments, or on certain days. One of those metrics is word variety by alignment, and, across all of my games on MU, there is a significant difference. The sort of situation I am trying to prevent is something like “I made a tool that scraped every post has ever made, ran a word choice analysis, and found that they’re are absolutely mafia because their word variety is far, far too low this game.”
To reiterate, I was never lobbying to ban the use of statistics, but rather the use of tools that allow for both rapid/automated data collation and analysis. To extremify the example, if I created a program, Wolf Detector 6000, that could accurately determine a player’s alignment given a large of dataset with 99.999% certainty, it quite literally ruins the game of Mafia in my view because once everyone else catches on to the reliably of such a tool, there’s no more game to be had unless every game is an anon game (which may not work forever). The Wolf Detector 6000 does not exist, but if you can analyze my word choice variance by alignment, and apply to ongoing games in real time, it would in practice be the Wolf Detector 6000. My alignment would be known, and this is not something I could trivially fix. Games with me as a player in it would be frustrating for wolves every time because either I’m a lock clear villager or outed wolf at some point in the game.
“But you could just fix it!” Yeah, great. Now I have to warp my game around making sure my posts have enough word variety to pass the smell test of some automated tool anyone could whip out at any point, and if I’m bad fooling it, I’m an Innocent/Guilty Child every game until I’m good at it. And what would probably happen is I would include a string of random words from the dictionary in every post. It would be stupid.
If you want to make an argument in game about my word choice variance, as long as you’re not automatically scraping posts and running them through a tool, it’s totally fair game. I just think that allowing both scraping and analysis tools is Pandora’s box. Not everyone is going to have a word choice variance tell, but if you can press a button and come back after making a cup of coffee, it’s a bit of an issue. And yes, statistical arguments are not always well understood, but my concern would be more their repeated use. If someone repeatedly uses a tool to make arguments that continue to be accurate, it becomes sort of a trust tell situation where people trust the statistical argument, not because they understand it, but because it’s been proven to be reliable. We must not create the Torment Nexus Wolf Detector 6000.
So, I also want to acknowledge that I may be misremembering the severity of my own word choice tell and could certainly be a Me problem that’s not generally applicable. I also want to reiterate that the Wolf Detector 6000 does not exist, and I don’t think it would for a long time. I do believe that some players will have tells that could be detected if you run the correct sort of analysis on their corpus of game posts, and that allowing and/or encouraging someone to try finding those players and the right sort of analysis is not a great idea for the game’s health. There is a point where it becomes obviously problematic, and it’s safer to ban the discussion of arguments based on automated collection and analysis of large datasets. Banning their use is unenforceable, but like I said in my previous point, if you have to dress up your statistical argument in a way that makes it seem like it’s using a much smaller sample size and using more of a bespoke process, the warping effects it can have on the game largely go away.
2 Likes
so I think one thing that’s being missed here that I’d like to discuss is that although I, for reasons I have stated and have been further expressed and expanded on by zugzwang/ranmilia/lumi/may/etc., am largely for not banning tools that automate data collection, because I feel this is an insanely wide umbrella that is unenforceable, I am notably not against banning discussing the usage of such tools
like
I personally think it’s far less central to the game of what Mafia is to have somebody be like “hey, ur a wolf, I used this open-source tool, feel free to check my work” than it is to have them make whatever case they were gonna make and not bring the tool up
so like I’d be more likely to support banning discussing the usage of outside data-collection programs than I would to support banning them wholesale, lol
it’s deffo a complex situation if you wanna get granular with things, which I think we do gotta do because wholesale “no outside programs” is still something I think is extreme and limits advancements that could help everybody long-term
I’ll see about drafting something up as a possible ruleset later
2 Likes
Either automated statistical analysis of large data sets is a waste of time for Mafia, and we lose nothing of value by banning it, or it’s not, and we should probably ban it. The Wolf Detector 6000 obviously trivializes the game, and where the line between “useful but not problematic” and “problematic” is on the spectrum of tools is I don’t know, but the point is there is a line. If what you tried during Champs had worked, yeah, I would’ve said it was unfair. It’s not compelling to me that your justification is you’re a slow reader, and it’s especially not all that compelling given that you went on to win Champs anyway lol.
Ranmilia in the Discord said " [it] creates slippery slope where if they are legal then everyone who wants an advantage (wolves trying to hammer at a deadline) become soft required to use them" regarding programs that post for you, and similarly reasoning can be applied to automated analysis tools. If they end up being worthwhile to use, players would be foolish to not use it. It would transform from a thing Lumi can do because she’s smart and a nerd to a thing everyone has to do because it’s good at finding wolves.
And last thing that I touched on again that I want to emphasize is that I think in practice, tools would probably only be able to determine the (likely) alignment of players who were polarized in particular ways, and that really sucks especially if the use of such a tool proliferates. Some players who happen to be outed by a tool just don’t get to really play werewolf anymore. Like it would suck if someone was deep in a game and they got outed because someone ran 300 of their games into a machine that popped out “wolf”, and people knew the machine was accurate 51% of the time. It could bring on a bunch of extra scrutiny on to them, and the problem is it wouldn’t really be by a player in the game. Like yeah whoever used the tool is its mouthpiece, but it’s not actually something from their brain, it’s the output of a program.
1 Like
This is where I point to Lumi’s entirely apt “moving target” thing - if you get told in game X you were caught for reason Y, then in game Z you’ll probably make an effort to not do that again
This is true regardless of whether reason Y was found by just a user or a user using a program to analyze simple publicly-accessible data
So I don’t really buy the idea that it locks polarized people out of playing entirely
1 Like
Tbh my opinion has changed a bit after reading through the replies and realizing some of the gaps in my logic
I can’t fully flesh out my thoughts on mobile atm though
I do think allowing tools on a case-by-case basis might be the way to go, but we will definitely run into some subjective decisions on the questions of “will this program give someone an unfair advantage” and “does this go against the spirit of the game”
I was thinking maybe we could have a list of Mod Approved Programs that are both okay to use and mention publicly (like multi-iso and vote compilations)? And try to make these programs as accessible as possible so everyone is aware they exist and has the option to use them
If we were to ban other programs that we deem DO give an advantage, we can’t really enforce it if they do choose to use them, no, but we can enforce it if the tool is mentioned in the thread. It’s kinda like how we deal with angleshooting lol - it’s banned, but we can’t exactly stop people from doing it. What we can stop is people bringing it into the game
8 Likes
that makes things easier
1 Like
and yeah as I said I think this rule is fair, and in conjunction with a re-tooling of rule 13 could be worked into a relatively solid piece of site policy
1 Like