Something has turned up at work

Uh oh.

Something has turned up at work, and I might not be able to update this blog as much as I would like to.

Of course if IM Bart or IM Bug can convince other strong players to participate in another game then I might reconsider 🙂

Thank you for your patience and understanding if you pardon the terrible cliche …

I’m Seeing The Same Scam Ads Again And Again

Well, not much luck with my efforts to get more readers into Spider Solitaire. I’m sure IM Bart and IM Bug will appreciate someone else stepping up to share the load.

Maybe I need to take a leaf from Facebook’s book (yes I know!) and start thinking of creative ways to encourage more participation in the Royal Game. What would Zuckerberg do?

What Would Zuckerberg Do?

It’s been a busy weekend

It’s been a busy weekend – but I’m not complaining 😊

On Friday my younger brother (the one without a Fields Medal) came over from Sydney with his partner and two children. I have also taken leave from work, starting on Friday and finishing the following Friday (8th April). Someone else and his family was supposed to come along for dinner, but the Coronavirus gods decreed otherwise.

But it’s not all doom and gloom with the Adelaide Crows winning an epic showdown against their bitter rivals Port Adelaide, and my niece has shown some promising signs as an artist. Over the past few years, she’s getting pretty good with happy stars and now she is moving on to bigger and better things, including Spider Solitaire.

My younger brother and his family are leaving this afternoon. After that it’s back to round 3 of the first game in our 5-point match with a Backgammon doubling cube showing the number ‘8’ face up. No pressure!

It’s Official – Many Spider Solitaire Programs Are Biased

It’s official. I believe many Spider Solitaire programs are biased – both for mobile phone and PC. Biased means if a player wins too many games, then the software will stack the cards against him in subsequent deals. Apologies if this comes across as a broken record, but “spider-software-bashing” is what inspired me to create this blog in the first place.

Let me state at the outset this is only my belief, and it comes with a number of caveats. I won’t expect regular readers to agree with my POV, but at least they can hopefully see where I am coming from. Following the rot13(erirefr fuvg fnaqjvpu) principle, I will start with a large number of caveats, then bring in the good stuff, and finally summarise these caveats. I assume we are always playing at the four-suit level (although it is possible to perform similar experiments with less than four suits).

The Caveats

  • Firstly, I don’t expect regular players to be able to reproduce my results. You will need a very good aptitude at the game. In particular you must be able to win practically every game at the four-suit level with enough perseverance and skill. Fortunately undo/restart is permissable (but not reject). Once you reach that ability you also need a “burn-in period” i.e., play enough hands to let the software deduce you have a good very aptitude for the game.
  • Secondly, I don’t have very many data points. The point of this article is to merely explain why I believe many programs are biased, and I am not claiming conclusive proof. Ideally, if time permits, I would prefer to play 100 games instead of 10. Oh, did I mention I am returning to work on Monday? In any case, if you are intelligent enough to play decently at the four-suit level then you are intelligent enough to draw your own conclusions.
  • I have no convincing explanation of what a software developer would gain by rigging the RNG.
  • A conclusive proof will entail some detailed knowledge of statistics (e.g. what is the meaning of statistically-significant, Komolgorov-Smirnov test, etc). I am nowhere near being in a position to submit a paper to Journal of the American Statistical Association.
  • I tested this on an earlier version of Ninja Monkey’s code, not the one I’m currently using now for the current game.
  • If you are stuck at 10-20% win rate then the bias is either non-existent, or doesn’t concern you. If you are winning above 20% then this is probably relevant. If you are serious about improving your game then all bets are off: your win rate is irrelevant and this is most definitely worth paying attention to.
  • I write this with a heavy heart – one of my readers plays a certain online server, which I now suspect to be biased.

Now Onto My Actual Claims

The certain online server in question is this one. Initially I thought this was kosher after some “experimentation”. However, one of my recent losses cut a little too close to the bone. I then made a serious effort to win 10 consecutive games for the first time – using RESTART if necessary. Note that the software forbids undoing 2 or more moves in a row (and single undo is illegal after dealing a row or removing a suit). I immediately sensed something was off.

I did manage to win all 10 games, but most of them were practically unwinnable without undo/restart. And not for lack of trying. Needless to say, if a Spider Solitaire program penalises a player who wins a few too many with undo/restart, then you can guess how it will treat a player who wins a few too many without these privileges.

Bad Beat

Listen To The Beat

As you might have guessed, I recorded the identity of every card in each of the 10 hands and ran Ninja Monkey’s algorithm. NM played each hand 50 times, for a total of 500 hands. Only for 2 hands did NM manage to win at least once. Whereas for random hands, NM should practically always win at least one in 50. I had similar results for a number of different programs.

Admittedly I may have tilted after a few games and failed to play to the best of my ability. But if anyone accuses NM of doing the same to corroborate my claims I don’t wanna hear about it.

I should point out one or two hands were grannies. Not exactly walkovers, but I would expect even my grandma will have a legit chance of winning sans undo. The term GRANNIE was apparently coined by a Scrabble expert – if you get all the power tiles then even your grandma can’t rot13(shpx) it up. In any case, one of my victories can safely be described as an EARNING.

My gut feeling says the software company wants to “equalise the players” so that (ideally) every single hand is either won by every player or lost by every player. Hopefully I don’t need to explain why this is a very bad idea.

Since it is beyond the scope to discuss the detailed mathematic specificity (with apologies to Kevin Rudd), I will simply present the initial game state for each of the 10 hands. I recommend the reader play the hands out for himself – first attempt to win without undo/restart, and if that fails then see how easy/difficult it is to win with undo/restart. The reader can also perform a control experiment by dealing 10 random hands. For extra credit: try to guess which two hands gave Ninja Monkey at least one victory in 50 tries.

List Can Be Any Colour As Long As It’s Black

As a result of this experiment, I have personally black-listed a number of Spider Solitaire programs, both for PC and mobile. From now on I only accept recognised programs like Microsoft Windows, or programs with source code (e.g. a GitHub project).

Games that “look and feel slick” are a red flag. Most of the bad programs I have encountered go beyond the basic “six-week-internship-prove-you-know-Javascript” thingy. While it is true that correlation is not causation, I would rather put my money on something that looks tacky instead of slick. Yes, the tableau cards do not flip automatically which is admittedly stupid, but at least you know what you’re getting. And the developer seems like a cool guy – anyone who’s willing to publish their name like that is almost certainly kosher. I found two look-and-feel-slick mobile phone apps, both dodgy and both had a similar “auto-complete” function when every column was either empty or reduced to a single straight-flush of any length. They’re probably from the same company.

Conversely if something looks like a basic implementation of Spider from someone who is learning the ropes of version-control in GitHub, then it is reasonable to assume there is no effort to punish a player who wins too often.

Yes, I hear you say, what possible motive do software developers have for punishing players who perform too well? True, not all Spider apps offer players a chance to enter competitions for “free” provided they sign up for PayPal, give away credit card details, their home and work addresses and the number of times they use the toilet every day. I was asking the exact same question and don’t have an answer. What I do know is that my numbers don’t lie – if Ninja Monkey says something is off, then Ninja Monkey says something is off.

IMHO, the onus should therefore be on software developers to prove they are kosher. Guilty until proven innocent if you pardon the inversion of a terrible cliché. If you can write a decent half-page essay on poker player Veronica Brill without help from Google, then you will certainly know where I am coming from.

Conclusion

I have decided many Spider Solitaire programs are not worth playing for both mobile phone and PC and I have blacklisted a number of those. But I wish to reiterate the fact this is my opinion only and my claims may well be difficult to substantiate for average players. Do not expect to see my name in lights if you subscribe to Journal of the American Statistical Association. But at least the reader has something to go on if he wishes to investigate for himself.

Finally, I wish to apologise if this comes across as a long-winded rant. But I believe this topic was worthy of more than the usual number of paragraphs.

And if you believe that some program is biased then you can also sit back and listen to the beat.

My Perilous Journey Into The GitHub Underworld

POP QUIZ: What could be more fun than me, two friends reading my blog and three blobs from the hit game Among Us participating in a multi-player game of Spider Solitaire?

Answer: Embarking on a perilous journey into the GitHub underworld.

Some time ago, I discovered that an “intelligent random move algorithm” can get a lousy win rate playing the Royal Game without rot13(haqb). By lousy, I mean around the vicinity of 6% – but lousy also means better than 0%. The basic gist of my algorithm is to repeatedly apply the following heuristics when choosing a sequence of moves:

  • basic tree search (i.e. examine N random different options, choose the best one)
  • static evaluation function (rewards in-suit builds, turnovers, moving suits to foundations etc)
  • multiple-move look-ahead, stopping as soon as at least one new card is turned over.
  • super-moves (a term borrowed from Freecell) to traverse the search space more efficiently

Apparently, this is enough to get around a 6% win ratio on random deals. I haven’t thoroughly tested this on a large number of data points, so this should be taken with a pinch of NaCl. But why not enlist the help of other Spider Solitaire or programming enthusiasts to boost the win rate even further?

So, I’ve decided to take the plunge and sign up for a GitHub account. Today, I have just created an account and uploaded my first project. My mad coding skillz aren’t nearly half as good as my mad Sudoku or Spider Solitaire skillz so hopefully I can keep my faux pas count to a minimum 😊 At least my actual job has provided me with some experience working with code repositories so I’m not totally out on a limb.

The current state of my project is a bare-bones AI, with plenty of room for improvement – a deliberate choice on my part. For instance, I explicitly stated in the readme.md file that one possible “to-do” is constructing a test suite to sanity-check that the AI scores a high/low/zero win rate for an easy/hard/impossible hand. I could have put in the effort to make the best possible AI before releasing it into the wild, but that’s not exactly in the spirit of collaboration with external programmers.

I already know at least one work colleague with a GitHub account and a number of “fun colleagues” from the Cracking the Cryptic discord server. With Christmas holidays around the corner (not to mention some flex leave arrangements) I should be in a good position to devote some serious time to this project.

I believe a good way to get some traction happening on my Spider Solitaire project is to return the favour for other people on GitHub. If I can spend some time “building reputation” (if you will), then I might find out that rot13(Xnezn nva’g n ovgpu) after all.

I have managed to make my first pull request for someone else’s code – a minor bug fix for a Spider Solitaire server written in JavaScript. There really should be a cheevo for submitting a meaningful pull request for someone else within 24 hours of creating a GitHub account – even if it doesn’t get accepted. But I’m sure there are worse injustices in the world I could wax lyrical about.

Interesting times ahead, watch this space 😊

Bart Wright and Schistocerca Americana awarded IM Titles

It’s official: after achieving victory on the Four-Suit version of Spider Solitaire sans rot13(haqb), with one impostor trying to rot13(shpx) things up at every opportunity I have decided to award both Bart Wright and Schistocerca Americana the title of International Master. It’s a bit of a misnomer in that we don’t get to travel all over the world to play in tournaments and prove we can mix it with the best – but I will stick with the IM title because it is considered the next level below GM and both players certainly displayed an extremely high level of skill. The other reason for the IM title is because it allows Spider GM to showcase himself at his brilliant worst. If you would like to guess the punchline before reading on, I have inserted the usual spoiler blocker.

In Chess, all titles are obtained by playing in enough tournaments and getting a sufficiently high score against decent opposition. Players can achieve “norms” in individual tournaments and you need enough norms to prove you didn’t get lucky in a single tournament. Your rating also needs to be sufficiently high. The good news is you don’t lose your title after a dip in your rating (or a string of bad tournaments), so you keep your title for life – save for something incredibly stupid, such as an arbiter finding compromising images of Stockfish recommending the best move on your mobile phone. This is fair enough: if there was no guarantee of keeping your title there would be much less incentive to achieve it in the first place. A GM title implies that your ability/talent is enough to mix it with the world’s best, provided you are willing to put in the time and effort and life doesn’t get in the way. An IM title is pretty much the same thing except “the next level down”. Then you get Federation Master titles and of course you can also add a W-hook (using the Scrabble vernacular) for female players. Hence WGM, WIM and so on.

Unfortunately, IM Schistocerca Americana is a bit of a mouthful even for somebody who recently composed a work-related rap song to the tune of Million Voices by Otto Knows. Therefore, he gets the indignity of being renamed IM Bug. And Bart Wright suffers an even worse fate:

Game On (27 June 2021)

Continuing from last week:

This is our cheat sheet so far:

One possible game state that can be reached from the start is the following:

Thanks to Bart Wright and S.A. for their input this week. Unfortunately, I’ve been busy with other stuff. To be more specific, I have discovered a truly remarkable idea for solving the problem of cheating in online chess tournaments. Unfortunately, this blog is not large enough to contain it. Besides, the tagline of the whole blog is “Spider Solitaire, the whole Spider Solitaire and nothing but the Spider Solitaire”, so I would also need to find some obscure connection to justify including this idea. On top of that, some of the comments failed the TL;DR test but the main gist seems to be “we can turn over new cards in Columns 1 and 6”.

S.A. gets bonus points for mentioning that we soon get to find out if Microsoft Solitaire allows scores to go into negative numbers, or whether the score remains at zero for the remainder of the game (and undo becomes free). I guess there is also the third option that the game terminates after 500 moves and we never get to find out if this game is winnable. Let’s hope it doesn’t come to that. But at least we have a “cliffhanger” which means Bart, S.A. and Sebastian are morally obliged to read the next few posts. Win-win for everybody 😊

Since we know the next deal is J6254A388K, it is relatively straightforward to determine the next unknown card in columns 1 and 6 starting from the above game state (NB: the next unknown card is determined by the cheat sheet, not the screenshot of the current game state).

Our luck is in: it is relatively straightforward to obtain a void in either Column 1 or Column 6 after dealing a row of ten cards. And yes, we have our first Nine 😊

It is not hard to see we have three guaranteed empty columns – so we should be able to determine the identity of almost every face down card. If you have played many games with undo, you are probably familiar with this pleasant situation.

QUESTION: What is the minimum guaranteed turnovers, starting from the above game state (score=109), playing with undo but not dealing any more rows from the stock?

BTW, in other news, Big Shiny Red Question Mark has been fully reinstated with all rights and privileges therein (thanks to Schistocerca Americana for the heads-up). But I’ve been told by a reliable source that BSRQM is still recovering from the mental scars of being Rick-Rolled. I guess there are things that 100,000,000,000 rupees can’t buy.

Awesomeness has been achieved!

I have finally achieved awesomeness! My Spider Solitaire Sudoku puzzle has been featured on Cracking the Cryptic.

For those who are interested in the puzzle only, here is the grid: if you’ve played any Spider Solitaire the rules should be guessable – and if you get a unique solution then you know you’ve guessed correctly 😊 But if you’re interested in the back-story then please read on.

If you follow this blog regularly, you are probably aware of a paper I published some time in 2019. I showed that a particular Spider Solitaire server was biased: if you win too many games then future games will have the cards stacked against you – and one could “prove” this using Statistics.

I use quote marks because the nature of Statistical testing always implies some degree of uncertainty. For instance if you are 95% confident of a hypothesis, then there is a 5% chance you made an error. But it is commonly accepted practice. If your experiment is sound and you get a sufficiently small p-value then go ahead and publish it anyway. You may be wrong, but – to put it in Poker terms – your results pretty much force you to call all the way to the river. If you are beat then you are beat.

Of course, getting the results you want is only the first step. We all know the academic publishing model is broken. The peer review model is hopelessly flawed. At best, peer review is based on good intentions and met the demands of research scientists 30 years ago – but certainly not today. It already takes long enough to get accepted into a mediocre journal, or even the dreaded arXiv. If you’re that desperate you might be willing to spell arXiv backwards. And don’t get me started on predatory journals. I won’t describe the ills of academic publishing in all its gory detail. Someone else can probably explain it much better than I can. In my case I ended up publishing into a high school journal. Parabola from UNSW to be exact.

But at the end of the day, publishing is essentially “a way to prove or showcase your research skills”. Once you complete your thesis (or minor thesis, 3-month vacation employment, etc) and may or may not be a major component of your career depending on your employment. (It is true that my Spider Solitaire paper is not relevant to my job, but that has nothing to do with the Fundamental Theorem of Calculus.)

Still, the Parabola publication still wasn’t entirely satisfactory. My paper wasn’t truly a publication. It was a story. I wanted to tell a story about how a certain Spider Solitaire was broken. There is nothing intrinsically wrong with Parabola (with the possible exception of some really lame comics and puns), but try telling that to the average Joe Bloggs with an average job, has little aptitude for mathematical puzzles and swears by Nova FM. In fact, telling this story was the original motivation for me starting this blog in the first place.

Scientists don’t have a way of getting their work recognised. They have no way of “controlling the narrative” if you will. I can publish a paper in some journal. Or I can post something on a blog and have all the scientific evidence to back it up. But how many people are going to read it, let alone believe it?

Enter Cracking the Cryptic.

You may have already guessed I am a fan of CtC (not necessarily because of this blog!). I was vaguely aware of it last year. It seemed to be massive in the UK.  I tried one of the harder puzzles. Solving it was beneath my dignity – after all I scored a silver medal in the 1995 International Mathematical Olympiad. Okay I get it. There’s a pandemic going on. People are struggling in the UK. Some viewers have even commented on YouTube how watching episodes of two people solving Sudoku puzzles helped their mental health issues. I’m living in Australia not the UK. Australia really is the lucky country, so who am I to judge?

I then stumbled on this puzzle by Lucy Audrin.

This is a “Sandwich Sudoku with a twist”. Before solving the puzzle Mark briefly mentions Lucy’s website and eventually finishes the puzzle in just over 15 minutes.

You read that right. Lucy wanted to draw attention to her website. All she had to do is submit a half-decent puzzle to CtC and Mark will take care of the rest. To be fair her puzzle is more than half-decent and a good illustration of how one can keep the puzzles fresh by tweaking various rulesets (such as thermometer, anti-Knight, XV, arrows etc). If Simon and Mark only did classic Sudoku every day of the week, CtC would have finished long ago. I should also mention that Lucy can write much better stories than I can!

Great – if Lucy can draw attention to her website then perhaps I can do the exact same thing with Spider Solitaire.

This was much harder than anticipated.

It would surprise nobody if I claimed I could construct a correct Sudoku puzzle with a unique solution and Spider Solitaire theme. There was one obvious hurdle: if I submit my puzzle and it gets rejected – then good luck trying to resubmit the same puzzle a second time. I decided to play it safe by first submitting “test puzzles”.

It was a long process. Essentially I needed to “play the networking game” and gradually build up reputation. I spent a significant amount of time testing puzzles by other setters, joining the Discord server and chatting, signing up for Patreon, creating my own puzzles, etc. I submitted the above Spider Solitaire paper to the Discord a few months ago, but eventually realised that was not the same as submitting directly to CtC (submitting to Discord only means CtC have permission to do it, if it gets nominated). Yes, networking really did make things a lot easier in the long run. If you play nice and do all the right things then eventually people will help you when you need them to. If Sudoku is your thing then I heartily recommend you join the discord server. Great people, great puzzles, great jokes and cultural references. Occasionally somebody may attempt to pull off a rick-roll. What’s not to like? 😊

I emailed CtC my puzzle earlier this week and finally my luck was in.

So there you have it. If you follow my blog regularly, then I hope you enjoyed the journey as much as I did. Until next time, happy Spider Solitairing 😊

Spider has made it on CTC!

Spider has finally appeared on Cracking The Cryptic! Unfortunately it’s not the right type of spider we know and love. We all know how many words a picture is worth so I’ll just do a screen dump and let you judge for yourself.

I believe every man dog and millipede on the planet must have heard of CTC by now. If you’re not familiar with CTC there’s always Google Search. If you’re not familiar with Google Search you can apply recursion and do a google search on Google Search. Bad jokes aside, this is easily one of the best Sudoku puzzles I have ever come across –

Yes, it’s a Sudoku. With no given digits. Not one. Just a bunch of lines that look like a spider. Plus a black dot between row 9 column 4 and row 9 column 5.

Lucy Audrin has set a number of puzzles for CTC and she also has many interests outside of Sudoku. She is definitely one of the Awesome People, and the world could do with a few more of those!

Oh yes, I should probably mention the rules: Normal Sudoku rules apply. Digits on a “thermometer” must strictly increase starting from the bulb (for example 13678). Some thermos share a common bulb. The black dot indicates two numbers in a ratio of 1:2 but you don’t know which cell is twice the other. It turns out this is enough to enforce a unique solution.

HINT: for those unfamiliar with Thermometer Sudoku, one of the thermometers is length 9, allowing you to enter nine digits immediately.

If you know your Kropki Sudoku, this puzzle has no “negative constraint” i.e. some cells can have consecutive digits or digits in 1:2 ratio despite the absence of a black or white dot.

Let me know if you enjoyed this Sudoku 😊