Annotate as you go

There's a counter-constructive saying: a project is dead as soon as you add documentation (Aeschylus, I believe).

This could not be more incorrect. Whereas it is true that writing documentation on an evolving project will quickly result in the fresh documentation becoming quickly invalid, it is a planning truth that writing documentation once a project is finishing is impossible because there are a hundread and one more pressing issues. Therefore, adding docstrings to each function, method and class in Python as one goes along is by far more advantageous. Once this is done, however this information needs to be transmuted into documentation. Here is how once can set up ReadTheDocs without falling into a few traps, as the documentation generator Sphinx is ironically weirdly documented and should be done ideally early on, so one knows what mistakes one's making.

JS in Colab

A Jupyter or Colab notebook has two sides, one is the Python kernel, which may be running on a remote machine, and the front-end running in one's browser. The JavaScript in the browser and the Python kernel as a result may be on separate machine, yet it is possible to make them dialogue. However, this differs between Jupyter and Colab, the latter being more restrictive. I have found this difference problematic and even though I may not be fully versed in Colab functionality I want share some pointers, discussed below. Majorly:

• Colab diverges greatly from Jupyter in terms of JS operations.
• JS code injected into Colab is sandboxed within each cell.
• There is no requireJS in Colab cells or window.
• Imported modules have to be external to Colab/Drive.

Things I wish I had known about Raspberry Pis

Raspberry Pis are very fun to play with and are relatively cheap —I believe I have 6 or 7 Pis running across the house. However, very often I learnt a thing or two that I wish I had known beforehand —sometimes rather fortuitously. So here is my list of top "in-hindsight" tips from the perspective of a seasoned coder with no prior electronics experience.

From cartoon to interactive infographic –the sane way

Making cartoon representations (technically vector graphics) in Adobe Illustrator is very fun, whereas the very idea of making a cartoon representation via line plots with Excel, Matlab, R, Plotly etc. would make anyone insane even just thinking about it. Luckily Illustrator images can be coloured based on numerical data in an automated way... without being manually plotted in Excel. Here I discuss exporting the vector graphic and modifying it with D3.js in a Jupyter notebook.

Convert Python docstrings to GitHub markdown readmes

This is suited for very small projects were read-the-docs is way overkill, however for most project that will be more suited — see how-to post on setting Sphinx for RTD for more

The Greek philosopher Epictetus said that a day reverse engineering a piece of code saves you half an hour reading the documentation. A maxim still valid to this day. Nevertheless, documenting code is important. With PyCharm and the push towards typehinting in Python writing docstrings is fairly simple. However, getting docstrings into the readme.md of GitHub is not straightforward the first time round. Hence, I wrote this simple guide to doing so.
Do note that for medium/big projects, using ReadTheDocs is recommended over this hack: ee the guide to setting up Sphinx for ReadTheDocs

Toasty CSS with BS4

In Bootstrap 4 you can have appear small alert-like rectangles, called toasts. However, getting these to work like notifications on top of the page in the top right is not trivial as it requires some CSS trickery. Here is what is required.

Pictograms with Plotly and FontAwesome

Plotly is one of the most powerful graphing packages for Python, JS and Julia. The cool feature is that the graphs are HTML bases with interactive graphs as opposed to a static jpg. There are several graphs that are missing, one of which is a pictogram. It's not a very silly graph, but  Luckily a pictogram is easy-ish to make.

Jupyter notebook progressbar

I have this rather handy wee piece of code I'd like to share: a Jupyter notebook Progress bar.

Wikipedia datamining

There are several online sites that can be data-mined to reveal really nice trends, top-10s and topdown summaries. Twitter is the archetype site for this, thanks to hashtags making an easy job for anyone wanting to investigate trends. I prefer Reddit for datamining specific trends as it powered by folk having arguments on topics they are passionate about as opposed to ideas of celebrities, corporate spokespeople and ФСБ agents. eBay is also fun as it reveals what people are willing to pay for things. But the best source of data, even for other datasets, is Wikipedia. Not only to read up on things, but also to get data for things within a given "category".

Exporting Jupyter notebooks with Plotly graphs

If it is a small project or analysis, I opt for Jupyter notebook rather than an IDE such as PyCharm, which is great for large projects, but not such much for a small analyse as go project. Plotly is my goto for graphs —I proselytise about it. The advantage is that it is a wrapper for a JS library which allows interactive. However, in my system at least, using the plotly.offline.iplot plotter, when I export it as a HTML an error is thrown due to require not being set up correction. This is easily fixed.

An arrow between Bootstrap cards

Recently I wanted to add an arrow (as in the triangle at the side of a tooltip or popover) pointing one card to its neighbour. It is only a few lines of code, but oddly the solutions available online are overly complex and wasteful. So this is my barebone solution.

Uniprot XML and Python ElementTree

Biopython does not have support for Uniprot. The reason is because it holds so much data that it would defeat the point to introduce a complex standard that the user would have to try and remember and the best way is for the user to choose themselves what piece of data they want.

Here I discuss the best way to deal with Uniprot XML files using ElementTree, which is really nice, but awkward at times, hence why I talk about a few monkeypatches that help. If you do not wish to deal with ElementTree (say you want a really quick, bu messy fix) see my post about complicated dictionaries.

A failsafe decorator for a python class

Often a Python class may have lots of bound methods that may fail, but it is not really a problem. Here I present a tidy way to deal with catching the errors with a decorator.

How to deal with horridly complex dictionaries in Python

NCBI and Uniprot data is complex, so is understandably stored as byzantine data structures, which have rather consistent schema, but hard to decipher. Depending on workflow, XML files are can be stored as dictionaries or as ElementTree.Element instances. I will talk about both, here I will talk about dictionaries —elsewhere I discuss using ElementTree. These are easier to deal with in some cases, but you can spend ages trying to find the series of keys and indices required to find a given value.
Here I present a nice pair of Python methods to get a given key or value in a convoluted object of nested dictionary-like and list-like objects.

Protein dropcaps

To make a webpage stylish does not mean that it cannot be biochemically themed. In fact, to can have both thanks to protein dropcaps! using the protein alphabet made by Prof. Mark Howarth in Nature Structural and Molecular Biology in May 2015(available Here) in combination with a wee JS script I wrote you can be a stylish biochemist. The script along with documentation is available in my GitHub page: https://github.com/matteoferla/protein_alphabet_dropcaps.

Pi projects: a new year's resolutions

2018 is here. I have papers that I want to or have to write and endless tasks I want to complete, but life is not all about career-driven self-sacrifice, it is also about having fun and what better way than with physical computing. And more time passes the more Rasberry pi projects I find I want to do. So I thought I'd share my shortlist of what I think are the most useful.

Going solo on a paper

This month my (first) solo paper comes out, which was a big deal as it is, well, my first solo paper.
The short communications describes the web app Mutanalyst: a good/amazing/best/super online tool to help calculate mutational bias spectra especially with poor sampling —in case you were wondering what does shameless Search Engine Optimisation look like, there goes an example. The topic is straightforward as it describes a web program I wrote with a twist, namely it calculate the standard errors, which are dismal when sampling is limited. The weird part is submitting a paper without backing, hence the account of my saga here.

Vulcan birthday calculator

When is it my Vulcan birthday?
For the exact length of a Vulcan year check below.

Caculator

Please insert your date of birth or age and the Vulcan year estimate:

DOB day:
DOB month:
DOB year:
Alt: Age:
Vulcan year length:

Vulcan years: 124.1
Next birthday: 5 Jul 2016

How long is a Vulcan year?

I have found three dates:

1. According to Memory Alpha Gene Roddenberry gives a Vulcan year as 456 ± 33 Terran days, which would make it a freezing iceblock.
2. Memory Beta has different, more realistic values. The only sourced one is 266.4 days, which is still on the chilly side for 40 Eridani.
3. On Star Trek Star Charts (2002) page 58 the year lasts 248 days.
   
   

Additionally, we know the following information:

• Vulcan is a warmer planet than Earth
• Vulcan (T'Khasi) orbits an orange star (Alam'ak, 40 Eridani A, spectral class: K1V), which is much cooler than Sol, a yellow star (G2V)

Consequently, it should be around 200 days.
However, there is the possibility that the definition of year (tevun) might not be an tropical orbital period around 40 Eridani. If there were little axial tilt and a near circular orbit, seasonal difference would be less pronounced (Game of Thrones's years Westeros can be explained this way in conjuction with short Milankovitch cycles).
PS. If you have any useful figures or facts about the year length, please do share!

Phylogenetic clunkiness

Writing a bacterial phylogenetics paper is probably the most agonizing type of paper to write due to the clunkiness of the language. But it's not the jargon that make life hell, it's the missing jargon that make it hard. The quantity of jargon actually make life a bit easier, but the lack of some blatantly useful words makes it painfully clunky and stuffy.

Reading a phylogenetics paper is generally visually painful due to the frequent usage of italics and "quote" marks. A phenomenon that I once read described aptly as a "typographic circus" by someone on a Wikipedia talk page. The italics are for taxa described validly according to the bacteriological code, whereas quotes are reserved for candidate taxa which have not been validated yet —the bugs exist, but nobody has done the paperwork. Nevertheless, the aesthetic problem is trivial compared to the lexal ones…

One such problem is the lack of trivial names (common names).
Only a few groups of  bacteria have trivial names and for some reason in microbiology, unlike botany, scientific names are rarely "trivialised", a process of making an English word for a member of a taxa, for example, a crustacean is a member of the family Crustacea and a bromeliad is a member of the genus Bromelia.
The most notable exceptions are a proteobacterium (a member of the phylum Protobacteria), an enterobacterium (a member of the family Enterobacteriales) and a pseudomonad (a member of the genus Pseudomonas). This is a shame as it circumvents the repetitive clunkiness and stuffiness. Taxonomic names, after all, are labels for the group, which can adopt a metaphorical usage to denote one of its members, primarily due to the quirkiness of the Latin name: for example, a group of sheep is a flock, but calling a single sheep a flock would sound awkward, whereas calling a sheep an Ovis sounds less weird, just technical. However, the phrase "a member of the" becomes really trite really quickly, one alternative is to use the word sp. (species), but it is still a bit weird: is the supposed to be the indefinite article "an" before Ovis sp.? A further perk is that a trivial name often allows adjectives to be formed, which is much nicer to read than a noun used adjectivally or "of the".
This last note is related to another problem with the Latin grammar of the nomenclature: all higher ranked names are plurals in Latin whereas there is only one of them (there are several members). In Latin (and other romance languages) collective nouns are plurals, whereas in English they are generally singulars (grammatical concord), but can be treated as plurals in informal speech (notional concord), e.g. the Alps is a mountain range (grammatical concord), the band are playing (notional concord). It does seem like a minor nuisance, but it gets really knotty when describing relationships and last common ancestors… on the plus side though the pride in the "look at my fancy Latin" is pretty nice.

Life expectancy in Middle-Earth: elves and mortality tables

If Uruk hai drove buses the

war of the ring would have been short. 

A scary thought: I have an a priori chance of one in a thousand of dropping dead within next time this year (qx) in New Zealand, which is far more likely than winning the kiwi lotto (on a lotto day, 1:4M). Life is unfair, eh?

Luckily, I am not doing anything dangerous nor have any underlying conditions hence the a priori —although, I have not bought a lotto ticket either.

If I were in my nineties the chances change to a one in three or four per year, so the above figure is really low really. The reason is obvious, namely deterioration due to old age.

80% of the probability for young folk to die is due to external causes, which means accidents, homicide, suicide or war. For the elder folk it is due to diseases such as cancer. Boys do much more stupid stuff than girls, therefore for a woman of my age, the chances drop to one in three thousand.

The life expectancy is around 67 years in developed countries, but what would it be if one did not age?

Agerasia is the mythical state where one does not grow old. It is different from immortality as it does not include invulnerability: the probability of external causes still applies. As this probability does not change with age, age of death would follow a geometric distribution (and not a Gompertz–Makeham curve —thanks, Wikipedia!—) and the mean will consequently be 1/p. This means that on average a kiwi male would live 1,300 years, whereas a female 5,500 years.

In fantasy and science fiction, long-lived or agerasic humans and humanoids often appear, but those figures do not match the above…

In The Lord of the Rings, the dwarves had a life expectancy of 250 years or so the web informs me because they age slower than men. The elves do not grow old and lived about 1,000 years —Galadriel (Kate Blanchett) was probably one of the oldest and was a bit over 5,000 years. Middle Earth would just slightly more dangerous than New Zealand, but not by much. (In (ir)reality, elves are hardier than men, so the comparison is slightly unfair.) Possibly unsurprisingly, the US, on the other hand, is slightly more dangerous than Middle Earth.

In Iain M. Banks's Culture novels the folk of the Culture are said to live on average 400 years. The figure is so low because they get utterly bored after a while and kick the bucket, however one chap lived for 10,000 years, but he was rather bonkers.

In the bible, the patriarchs before the great flood lives for a long time. Their age of death are given which is really handy, but there are only ten of them and one, Enoch, did not technically die. Despite the small sample size, they lived on average 912 years, but the numbers are curious as they all died between 900 and 1,000 which is statistically impossible both under a geometric and a Gompertz distribution. The explanation that they get a "millenium crisis" at 900 does not fit as it is written they die of super-old age (which hits all of a sudden). Given that only Abel dies of external causes (1 in 10), it could be deduced that antediluvian Israel was much safer than New Zealand, Middle Earth or the US, but whatever rabbi wrote down the numbers wasn't well versed with statistics, so the bias is too strong for any conclusions.

Sources: Wikipedia, NZ life tables, some bible website, Iain Bank's Hydrogen Sonata and several unutterably nerdy LotR sites…