Geek's Code Corner

While Pandas and Seaborn offer very quick ways to calculate correlations and show them in a heatmap. Whether those correlations are statistically significant or not is omitted from those plots. Over the years I collected bits and pieces of code, like this, that turn out to be quite useful. Though them being scattered across a few dozen projects isn鈥檛 very convenient when I actually need them. So I’ll start to add some documentation and put them here with the tag Code Nugget, so they can easily be found by myself and others. Normally you can use corr_df = df.corr() to get a correlation matrix for numerical columns in a Pandas data frame. These in turn can be shown in a heatmap using sns.clustermap(corr_df, cmap=”vlag”, vmin=-1, vmax=1), leveraging SeaBorn clustermap. Easy, though the significance of those correlations isn’t reported. To get those you can rely on built-in functions and a bit more effort is required. from sklearn.datasets import load_iris from scipy.stats import spearmanr import pandas as pd import numpy as np import seaborn as sns import matplotlib.pyplot as plt from statsmodels.stats.multitest import multipletests iris_obj = load_iris() iris_df = pd.DataFrame(iris_obj.data, columns=iris_obj.feature_names) def get_correlations(df): df = df.dropna()._get_numeric_data() dfcols = pd.DataFrame(columns=df.columns) pvalues = dfcols.transpose().join(dfcols, Read more…

Bayesian analysis and probabilistic programming is fairly different from ‘regular’ analysis. So here I’ll take you through my first attempt at using PyMC3 to analyse sales data of KeyForge decks and explore the effects of COVID-19 restrictions and releasing new content. PyMC3 takes some getting used to, but at the end you’ll see there are some clear advantages of this approach! In this post we are looking at KeyForge sales, a collectible card game from FFG, which is in many ways unique. However, for sake of this post there are just a few things that matter; the game is sold as individual decks or starter kits with two decks (and some tokens to play the game). Each deck is randomly generated and unique, decks are supposed to be played as is, without changing the cards. Players are encouraged to scan a QR code included with each deck using the companion app to register it online. The number of decks that has been registered since KeyForge’s release in November 2018 can be found on their website. New sets are released every six to eight months with new cards and new mechanics, prior to each release there is a period where new Read more…

After seeing the Episode on Benford’s Law in Latif Nasser’s Connected (currently on Netflix), I had to test if this is true and if I could find a few datasets where it applies. And … I did! But before we get into details, let’s first explain what Benford’s Law is. Benford’s Law states that the leading digit (that is the first digit in a number, also called the most significant digit) in a lot of collections of numbers is not randomly distributed. The first digit will most frequently be 1 (in about 30% of the numbers as opposed to 11% if it were truly random), the next most frequent digit will be two, … and the least frequent will be 9 ( which would be the first digit far less frequent than you would expect). With other numbers decreasing in probability according to a log scale. The probabilities for each digit he proposed would fit this formula, P(n)=log10(1+1/n), and with a little python code we can quickly compute these and plot them out. import pandas as pd import numpy as np import seaborn as sns palette = sns.color_palette(“GnBu_r”, 9) benford_proba = ({‘digit’: i, ‘prob’: np.log10(1+1/i)} for i in range(1,10,1)) ideal_df Read more…

In an article I contributed to Team Bandit Gang’s website, data is shown about the number of games Pro Ranked players play at different ranks. Tough with the data that can be pulled from Gwent Masters assessing whether or not playing more will result in a higher ranking is difficult. So answering the question posed in the article, whether it is skill vs grinding games that will help you further, no definitive conclusions could be drawn. The main issue with the data released at the end of the season is that there is no good way to judge a player’s actual skill. You could use the number of games played and peak MMR as a metric for efficiency, but that isn’t necessarily a good proxy for skill. Playing all six factions would affect that score negatively, as well as players that have a very good performance but decide to play more casually with fun, less optimal, decks after achieving an MMR score they are happy with. While this could be improved by scraping players’ profiles several times throughout the season, assumptions would still need to be made how to translate this into an approximation for skill. While skill can’t be Read more…

To expand the number of notes/chords my RP2040 based MIDI controller can play we’ll turn two keys into modifiers. This way the other 7 buttons can actually each be assigned 4 different notes/chords (a default, pressed with modifier one, pressed with modifier two and pressed with both modifiers active). So we go from 9 notes/chords to 28 different options, which should expand the number of songs that can be played substantially. While at first glance this seems like an easy task, there are a few things that need to be taken into consideration. A MIDI device sends a signal once a note is pressed and one when it is released. In the previous post this was straightforward; each button was playing two notes one octave apart, and there was no overlap between buttons. So when a button goes down, the corresponding notes can start, if that button goes up those notes can be started again, easy … When assigning proper chords to buttons, things get more complex. Imagine a key that plays a C chord, once pressed a signal is sent to play notes C, E and G. If that key is released a signal to stop those notes is Read more…

Doing an electronics project where space comes at a premium? A regular rpi pico is to large? Have a look at the XIAO-RP2040 from Seeed Studio. Seeed Studio provided me a XIAO-RP2040 to review so let鈥檚 have a look at this nifty little board. The microchip is identical, though as fewer pins are broken out the size of the board is reduced to about 1/3 of the original Pi Pico. Despite its diminutive size, the XIAO-RP2040 has a few things going on that are very interesting. There is a reset button, which means no more un-plugging and re-connecting in case a bug in the code causes a crash. In addition to the onboard LED, there is a NeoPixel RGB LED included on the board itself. Last, but definitely not least, the XIAO-RP2040 comes with a USB Type-C connector, making your projects future-proof. Getting Started The board came loaded with Arduino firmware, however loading CircuitPython was a breeze. Just follow the official instructions. In a nutshell, you need to be holding the boot button while connecting to a computer. The device will appear as a USB drive on your computer and you simmply drop the CircuitPython .UF2 file there and restart. Read more…