# articles

feed_id: urn:md5:c503e72c84adacc9a61e969997d49292
title: Synchroniser deux dépôts Git en local
uri: Synchroniser-deux-dépôts-Git-en-local
url: http://bfontaine.net/blog/post/2012/04/10/Synchroniser-deux-d%C3%A9p%C3%B4ts-Git-en-local
content:
    <p>Il peut parfois être utile de synchroniser des dépôts Git sur diverses machines en local. Par exemple, si l'on possède un compte <a href="https://github.com/" hreflang="en">Github</a>, et que l'une des machines n'a pas accès à Internet (c'est mon cas temporairement), on peut se synchroniser avec Github avec la machine connectée, puis synchroniser celle-ci avec la machine hors ligne.</p>
<p>Imaginons que l'on ai un ordinateur portable ayant accès à internet, accessible en local à l'adresse IP <code>192.168.1.20</code>, et un ordinateur fixe accessible en local à l'adresse IP <code>192.168.1.21</code>. Chacun de ces ordinateurs a un dépôt Git situé en <code>/home/toto/MonProgTropBien/</code>.&nbsp;Il suffit d'utiliser&nbsp;<code>git remote add</code>, pour ajouter un dépôt distant sur chacun des ordinateurs.</p>
<p>La syntaxe est la suivante (il faut être dans le répertoire du dépôt Git) :</p>
<pre>git remote add &lt;nom-remote&gt; &lt;nom-utilisateur&gt;@&lt;adresse&gt;:&lt;chemin-distant&gt;</pre>
<p>On tape donc la commande suivante sur le fixe :</p>
<pre>git remote add laptop toto@192.168.1.20:/home/toto/MonProgTropBien</pre>
<p>La commande est similaire sur le portable :</p>
<pre>git remote add fixe toto@192.168.1.21:/home/toto/MonProgTropBien</pre>
<p>Ensuite, il suffit de se placer dans le répertoire du dépôt Git, et d'utiliser <code>git push fixe</code> sur le portable (ou <code>git pull laptop</code> sur le fixe) pour envoyer les modifications sur le fixe, et <code>git pull fixe</code> pour l'in<wbr/>verse (<code>git push laptop</code> depuis le fixe). La commande <code>git add remote</code> ne doit être exécutée que la première fois.</p>
<p>Cette mani<wbr/>pulation permet également de synchroniser deux dépôts sur la même machine (utilisez la syntaxe <code>git remote add &lt;nom&gt; &lt;chemin-de-l'autre-dépôt&gt;</code>).</p>


feed_id: urn:md5:388b70573e8f8b45bd581e8b19c4c31e
title: Connectez-vous facilement à up7d
uri: Connectez-vous-facilement-à-up7d
url: http://bfontaine.net/blog/post/2012/03/08/Connectez-vous-facilement-%C3%A0-up7d
content:
    <p>À l’université <a href="http://www.univ-paris-diderot.fr/sc/site.php?bc=accueil&amp;np=accueil" hreflang="fr">Paris 7</a>, plus précisément sur le campus <abbr title="Paris Rive Gauche">PRG</abbr>, est accessible une borne WiFi dont le <a href="http://fr.wikipedia.org/wiki/SSID">SSID</a> est <code>up7d</code>. Pour s'y connecter, il faut avoir un compte <a href="https://cas.sc.univ-paris-diderot.fr/">ENT</a> (seuls les étudiants et le personnel en ont un). Le principe de connexion est le même que pour une borne <i>Free WiFi</i> ou <i>Neuf WiFi</i> : il n'y a pas de clef WEP ou WPA(2), et on tombe sur une page de connexion lorsqu'on y accède, nous demandant login et mot de passe. Il est assez fastidieux de devoir entrer son login et mot de passe à chaque fois, surtout que la page est parfois lente à charger, et nécessite de façon totalement injustifiée l'activation du Javascript.</p>
<p>Pour résoudre ce problème, j'ai développé un petit script en Ruby, <code>up7connect.rb</code>, qui automatise l'authentification. Lors du premier lancement, il demande le login et mot de passe à utiliser. Ces informations sont chiffrées et stockées localement. Ensuite, il suffit de le lancer pour qu'il vous authentifie. Si tout est bon, il affiche <q>Connection ok.</q>&nbsp;Si l'on est déjà connecté, il le précise. À noter qu'il faut avoir déjà sélectionné le réseau sur son ordinateur, le script vérifie d'ailleurs si une borne dont le SSID est <code>up7d</code> est bien accessible. Il nécessite Ruby 1.9 pour fonctionner, et s'utilise en ligne de commande (option <code>--help</code> ou <code>-h</code> pour l'aide.)</p>
<p>Au niveau technique, pour la connexion j'utilise le module <code>net/http</code> pour envoyer les données du formulaire originel à la bonne page. La page est en HTTPS, mais le certificat SSL n'est pas valide, donc je ne le vérifie pas (<code>OpenSSL::SSL::VERIFY_NONE</code>). J'utilise un user-agent personnalisé, et j'envoi un <i>header</i> <code>Referer</code> avec l'adresse de la page de login originelle, sinon je suis bloqué par une <a href="http://fr.wikipedia.org/wiki/Liste_des_codes_HTTP#Erreur_du_client">erreur 405</a> (méthode de requête non autorisée). Pour le stockage du login et du mot de passe, mon algorithme est le suivant:</p>
<pre>j1 = caractère au hasard dans [a-z]<br />j2 = caractère au hasard dans [a-z]<br />s = login + &lt;chaîne arbitraire&gt; + mot de passe<br />l1 = liste des codes des caractères de s<br />s1 = éléments de l1 joints avec j1<br />l2 = liste des codes des caractères de s1<br />s2 = éléments de l2 joints avec j2<br />pour chaque chiffre e dans s2 faire:<br />    e = caractère correspondant à 33+e<br />fin pour<br />écrire s2 dans un ~/.up7connect.conf</pre><p>Le déchiffrage se résume à faire la manipulation inverse à coup de <code>split</code> avec des expressions régulières.</p>
<p>Le script fonctionne sur les machines sous UNIX-like. Pour Windows, je n'ai pas pu tester, n'ayant pas d'environnement adéquat (<i>i.e.</i> un laptop sous Windows). Par défaut, le fichier chiffré contenant login et mot de passe est situé dans le répertoire courant sous Windows, et dans le répertoire de login pour les autres. Pour Linux, j'utilise la commande <code>iwlist wlan0 scan last</code> pour voir si le SSID <code>up7d</code> est visible, tandis que je ne fais pas cette vérification sous Windows/Mac/FreeBSD (je ne connais pas la commande correspondante pour Windows/Mac, c'est à venir pour FreeBSD).</p>
<p>Le script est actuellement en version 0.1, sous licence libre <i>GPL v3</i>, et <a href="http://bfontaine.net/files/ruby/up7connect.rb" hreflang="en">disponible ici</a>. Il fonctionne bien, malgré parfois une petite difficulté à déterminer si le SSID <code>up7d</code> est visible (il arrive rarement qu'il le considère comme invisible alors qu'il est bien là).</p>
<p><strong>MàJ du 11 mars 2012</strong> : Le script n'essaye plus d'utiliser la commande <code>iwlist</code> sous Mac, car elle n'y est pas disponible. Cette version est la 0.1b et remplace la précédente.</p>


feed_id: urn:md5:4eb89de66b4278b07d0bb8fb8e30c15d
title: Le faux problème du TDD brainless
uri: Le-faux-problème-du-TDD-brainless
url: http://bfontaine.net/blog/post/2012/02/23/Le-faux-probl%C3%A8me-du-TDD-Brainless
content:
    <p>Ce billet est issu d'une réflexion suite à la discussion de la <a href="http://www.meetup.com/paris-software-craftsmanship/events/48928732/">dernière rencontre Software Craftmanship</a> sur le <abbr title="Test Driven Development">TDD</abbr> <i lang="en">brainless</i>, <i>i.e.</i> le TDD où l'on ne fait qu'implémenter bêtement le strict minimum nécessaire pour valider chaque test, sous une forme suivante :</p>
<pre>def tests(): # mes tests<br />    assertEqual(my_func(entree1), sortie1)<br />    assertEqual(my_func(entree2), sortie2)<br />    assertEqual(my_func(entree3), sortie3)<br /><br />def my_func(entree): # ma fonction<br />    if (entree == entree1):<br />        return sortie1<br />    if (entree == entree2):<br />        return sortie2<br />    if (entree == entree3):<br />        return sortie3</pre><p>Le principe du TDD est bien d'implémenter le strict minimum pour passer les tests, mais de façon paresseuse, pas de façon bête. Autrement dit, on implémente de la façon qui nous demande le moins de travail possible. Le TDD <i lang="en">brainless</i> ne permet pas cela, puisque, avec la méthode donnée ci-dessus en exemple, on devra implémenter <i>N</i> conditions <code>if</code> pour chacun des <i>N</i> tests, ce qui est relativement rapide quand on a très peu de tests à faire, mais bien plus fastidieux quand on en a beaucoup. Justement, le TDD paresseux consiste à se dire que puisque la méthode est fastidieuse, il faut chercher un moyen de passer les tests de façon plus économique, ce qui entraîne nécessairement une réflexion (d'où la différence avec le TDD <i lang="en">brainless</i>) sur un algorithme permettant d'implémenter la fonction testée sans trop d'efforts. À la fin, on arrive donc à une fonction implémentée de façon minimale qui passe les tests, et qui (si les tests ont été bien choisis) fonctionne, contrairement à une fonction implémentée avec du TDD <i lang="en">brainless</i>, qui ne fonctionne que pour les cas de tests.</p>
<p>Tout ça pour dire que non, pousser la logique du TDD jusqu'au bout ne donne pas nécessairement du TDD <i lang="en">brainless</i>.</p>


feed_id: urn:md5:2662fd80adf6f99d9c3299d1eeedd180
title: Nouveau site de Paris 7 : Peut mieux faire
uri: Nouveau-site-de-Paris-7-Peut-mieux-faire
url: http://bfontaine.net/blog/post/2012/02/13/Nouveau-site-de-Paris-7
content:
    <p>L'Université Paris 7 (Diderot) a récemment <a href="http://www.univ-paris-diderot.fr/sc/site.php?bc=accueil&amp;np=pageActu&amp;ref=3949" hreflang="fr">refait l'apparence son site</a>. L'annonce dit :&nbsp;<q><i>Nouvelle ergonomie, nouveau graphisme, pour un site web plus lisible, plus facile d'utilisation, plus dynamique aussi</i></q>. Si l'ergonomie du site n'est effectivement pas trop mauvaise, même si on se perd un peu dans l'arborescence des pages (Essayez, à partir de la page d'accueil, d'arriver sur la rubrique consacrée aux stages, sans passer par la recherche. Bonne chance); par contre au niveau de l'accessibilité et des performances, c'est la catastrophe. Bien que le site prétend se mettre <q>progressivement en conformité</q> avec le <a href="http://references.modernisation.gouv.fr/rgaa-accessibilite" hreflang="fr">RGAA</a> depuis <a href="http://web.archive.org/web/20080913095553/http://www.univ-paris-diderot.fr/sc/site.php?bc=universite&amp;np=mentions_legales" hreflang="fr"><i>au moins</i> 2008</a>, on remarque que le champ de recherche ne comporte pas de libellé (<code>&lt;label&gt;</code>), et ce depuis <a href="http://web.archive.org/web/20061128185310/http://www.univ-paris-diderot.fr/cherche.html" hreflang="fr">au moins 2006</a>, époque à laquelle la page du service de communication était ornée d'un <a href="http://web.archive.org/web/20061128232402/http://www.univ-paris-diderot.fr/comm/communic.htm" hreflang="fr">beau gif animé</a>. Le site n'est évidemment pas valide <a href="http://www.w3.org/TR/WAI-WEBCONTENT/#wc-priority-1" hreflang="en">WCAG niveau A</a> (pas d'attribut <code>alt</code> sur des images qui servent de lien), et on notera que certaines parties du site ont une taille de police spécifiée en pixels, donc la taille ne bouge pas si on augmente la taille de la police via le menu du navigateur.</p>
<p>Sans surprise, le site n'est pas<a href="http://validator.w3.org/" hreflang="en"> valide HTML</a>, avec 196 erreurs, dont la majorité est en fait issue du fait que certaines URL contiennent le caractère "<i>&amp;</i>" au lieu de "<i>&amp;amp;</i>", son code HTML.</p>
<h5>Performances générales</h5>
<p>Au niveau des performances, <a href="http://www.webpagetest.org/result/120213_CE_37M1K/1/details/" hreflang="en">WebPageTest</a> nous indique que le <i>start render</i> est à 2 secondes et le temps de chargement total est de 7.2 secondes la première fois (respectivement 0.8 secondes et 2.8 secondes la seconde, avec le cache), pour un total de 54 requêtes. On note que lorsqu'on tente d'accéder à l'URL principale du site, <i>i.e.</i> <code>www.univ-paris-diderot.fr</code>, on est redirigé deux fois avant d'arriver sur la véritable page d'accueil. La connexion est <i>Keep-Alive</i> (bon point), mais aucune ressource (même les scripts, le HTML ou les feuilles de style) n'est compressée.</p>
<h5>Styles</h5>
<p>La page comporte 7 feuilles de styles, dont deux quasiment identiques (<code>site.css</code> et <code>sitefr.css</code>), non minifiées, et surtout pas adaptées aux terminaux mobiles (pas de <code>@media-query</code>, et des éléments de <code>900px</code> de largeur), la fenêtre devient d'ailleurs trop petite en dessous de 930 points. Cinq d'entre elles sont chargés classiquement dans le <code>&lt;head&gt;</code>, et deux autres dans le corps de la page (non valide).</p>
<h5>Scripts</h5>
<p>La page comporte 9 scripts externes, dont un qui n'existe pas (Erreur 404. Si on tente de l'ouvrir dans le navigateur, on tombe sur une page 404 personnalisé, qui tente d'afficher des images qui… n'existent pas), et <i>jQuery</i> version 1.6 (importé deux fois !) minifié (c'est le seul script qui l'est), ainsi qu'une version 1.7. À l'exception de quelques uns chargés dans le <code>&lt;body&gt;</code>, tous les scripts sont chargés en tout premier dans le <code>&lt;head&gt;</code>, alors que ceux-ci ne sont pas tous nécessaires au rendu de la page. On trouve par ailleurs un script <i>inline</i> téléchargé sur&nbsp;<i>EasyScript</i>, qui n'est jamais utilisé (il permet d'ouvrir une pop-up pour afficher une image), et du <i>Google Analytics</i> en bas de page.</p>
<p>Parmi ces scripts chargés en tout début de page, en dehors des deux nécessaires au diaporama, aucun n'est nécessaire au rendu, donc ce placement est tout sauf justifié. Dans les détails, parmi les scripts existants, on trouve d'abord <i>jQuery</i>, puis un script (copié sans citer <a href="http://www.guillaumevoisin.fr/developpement/tutoriel-realiser-un-diaporama-simple-avec-jquery#" hreflang="fr">la source</a>) permettant de faire des diaporamas, puis un script utilisant le précédent qui démarre un diaporama une fois la page entièrement chargée, et enfin un dernier qui n'a pas d'autre fonction que d'ajouter/enlever une classe aux éléments du menu survolés avec la souris (<code>:hover</code> ? Connaît pas). Notons qu'en dehors de <i>jQuery</i>, aucun de ces scripts n'est optimisé (les sélecteurs <i>jQuery</i> sont utilisés à outrance sans utiliser de mise en cache, on trouve de multiples mot-clefs <code>var</code>, et la taille d'un tableau n'est pas mise en cache lors du parcours de celui-ci avec une boucle <code>for</code>), et on trouve des incohérences, par exemple le script <code>menu.js</code> modifie à l'arrache les classes d'un élément (via une expression régulière), alors que <i>jQuery</i>,&nbsp;inclut&nbsp;un peu au dessus, propose les méthodes <code>.addClass()</code> et <code>.removeClass()</code>.</p>
<h5>Images</h5>
<p>Les images ne sont pas servies via un (sous-)domaine différent, donc chaque requête envoit les cookies pour rien. Les dimensions réelles de l'image ne sont pas indiquées dans le code HTML, ce qui aurait pu éviter un temps de calcul supplémentaire de la part du navigateur.</p>
<h3 style="font-weight:bold;color:black;">Proposition</h3>
<p>Inspiré par le concours <a href="http://webperf-contest.com/index-fr.html" hreflang="fr">WebPerf Contest</a> qui a eu lieu en 2010 et dont l'objet était l'optimisation d'une page du site de la Fnac, et suite aux multiples observations faites dans toute la partie supérieure du billet, je me suis lancé un petit défi dont le but est d'optimiser la page d'accueil du nouveau site de Paris&nbsp;7, en appliquant toutes les bonnes pratiques possibles de façon à ce que le site soit plus performant. Les règles sont les mêmes que pour le concours, l'apparence du site doit rester la même, toutes les fonctionnalités actuelles doivent rester disponibles (en l'occurrence, c'est plus simple, il n'y a aucune requête AJAX); Le but principal étant d'optimiser la page, mais tout ce qui peut améliorer l'expérience utilisateur (notamment au niveau de l'accessibilité) est un plus, du moment que ça ne prend pas des jours à réaliser.</p>
<h5>1- Récupérer la page d'accueil ainsi que tous les composants externes</h5>
<p>J'ai simplement utilisé Chromium pour enregistrer la page avec ses dépendances (images, fichiers CSS et Javascript). Certains fichiers n'ont pas été téléchargés (les fichiers chargés via des scripts, notamment), pour ceux-ci il a fallu les chercher à la main en regardant les erreurs 404 générées.</p>
<h5>2- S'organiser</h5>
<p>J'ai créé un <a href="https://github.com/bfontaine/up7-optimization" hreflang="en">dépôt</a> sur <a href="https://github.com/" hreflang="en">Github</a>&nbsp;pour tout versionner afin d'organiser le travail, et de garder un historique des optimisations. J'ai également créé deux sous-domaines (<i>static1</i> et <i>static2</i>) sur mon site en vue de les utiliser pour les médias (en l'occurrence, uniquement des images).</p>
<h5>3- Nettoyer</h5>
<p>Il a d'abord fallu nettoyer un peu tous ces fichiers, les feuilles CSS étaient par exemple remplies de commentaires. Ainsi, (presque) tous les commentaires des scripts, du fichier HTML principal et des feuilles CSS ont été supprimés. Certains scripts étaient inutiles et ont été retirés, tandis que les autres ont été fusionnés de telle façon qu'il n'en reste que 2 (sur les 7 originaux). Du côté CSS, on est passé de 7 feuilles de style à 5.</p>
<h5>4- Préserver la compatibilité</h5>
<p>La page originelle affiche une image aléatoire (parmi 5) en haut, ceci a été préservé, ainsi que la présence d'un cookie (j'ai simplement utilisé un cookie de session).</p>
<h5>5- Réduire</h5>
<p>La majeure partie de l'optimisation consiste à réduire, d'une part, le poids total des éléments de la page, et de l'autre le nombre de requêtes. J'ai optimisé les images PNG avec les outils <code>optipng</code> et <code>pngcrush</code>, tandis que pour les JPG, j'ai utilisé <code>jpegtran</code>. Ces outils sont lossless, i.e. il n'y a pas de pertes de données. J'ai également utilisé l'option <code>-quality</code> de l'outil <code>convert</code> (qui fait partie d'<i>imagemagick</i>) pour réduire (avec perte de qualité) certaines images qui n'avaient pas besoin d'être de grande qualité (les icônes, notamment). Les images statiques (<i>i.e.</i> qui sont toujours sur la page d'accueil, quel que soit son contenu) comme les drapeaux ou les logos Facebook/Twitter ont étées combinées en une seule, en utilisant la technique des CSS Sprites. Les feuilles CSS et les scripts ont été optimisés puis <a href="http://jscompress.com/" hreflang="en">minifiés</a>.</p>
<h5>6- Mettre en cache</h5>
<p>Les images ont été réparties sur trois sous-domaines différents; les PNG sur <a href="http://static1.bfontaine.net/up7opt/">static1.bfontaine.net/up7opt/</a>, les JPG et GIF sur&nbsp;<a href="http://static2.bfontaine.net/up7opt/">static2.bfontaine.net/up7opt/</a>, tandis que la <i>favicon</i> et les images de fond du diaporama ont été gardées sur le sous-domaine principal. Ceci permet de paralléliser les chargements et de supprimer les cookies dans les requêtes d'images (sur les autres sous-domaines). les feuilles CSS et les scripts sont également mis en cache.</p>
<h4 style="font-weight:bold;color:black;">Résultat</h4>
<div>Le <a href="http://labs.bfontaine.net/misc/up7opt/index.php" hreflang="fr">résultat est visible ici</a>. D'après WebPageTest, qui donne un score honorable de 95/100 (le site original n'a que 40/100), on a un temps de chargement de 3.6 secondes la première fois (2.2 secondes la deuxième), soit près de la moitié du temps original. Le <i>start render</i> est à 0.8 secondes (2.4 fois plus rapide). Il y a près de 100 éléments DOM en moins, le premier chargement demande 30 requêtes (-45%) et le second seulement 3 (-94%). Au niveau poids de la page, on passe de 1.2Mio à 534Kio (-65%) pour le premier chargement, et le second ne pèse que 302Kio (ceci est dû au fait qu'il faille charger une nouvelle image à chaque fois, l'image en haut de la page étant aléatoire).</div><h5>Remarques</h5>
<div>Je n'ai fait qu'optimiser le site au niveau des performances, je n'ai pas touché à l'accessibilité ni à la correction des erreurs dans le HTML, ni à l'adaptation au mobile, par manque de temps. Pour bien faire, il faudrait réécrire entièrement la page.</div>


feed_id: urn:md5:d8d84aa173177208db23167157cc7bed
title: Cachez les publicités sur Facebook
uri: Cachez-les-publicités-sur-Facebook
url: http://bfontaine.net/blog/post/2012/01/30/Cachez-les-pubs-sur-Facebook
content:
    <p>Facebook affiche des publicités dans une colonne sur la droite des pages. Pour les cacher, ajoutez la règle suivante à votre feuille CSS personnalisée :</p>
<pre>div[data-ad] {visibility:hidden}</pre><p>Pour Chromium, sous GNU/Linux, la feuille de style personnalisée, Custom.css, est située dans le répertoire suivant :</p>
<pre>/home/baptiste/.config/chromium/Default/User StyleSheets/</pre><p>Il est également possible de modifier la feuille de style utilisée par l'extension <i>Adblocks</i>, si vous l'utilisez.</p>
<p><u>Explication</u> : On sélectionne tous les blocs <code>&lt;div&gt;</code> qui contiennent l'attribut <code>data-ad</code>. En effet, Facebook utilise cet attribut pour stocker des données relatives aux publicités. La spécification HTML5 permet d'utiliser des <a href="http://dev.w3.org/html5/spec/Overview.html#embedding-custom-non-visible-data-with-the-data-attributes" hreflang="en">attributs personnalisés</a> dans le code HTML, qui commencent par <code>data-</code>. Twitter utilise le même système avec <code>data-user-id</code> et <code>data-tweet-id</code> (entre autres). C'est très utile lorsqu'on manipule le DOM avec Javascript.</p>


feed_id: urn:md5:0dd6e8b49fe07510ae26d7a3c492f28d
title: Créez vos propres fichiers pour Cowsay
uri: Créez-vos-propres-fichiers-pour-Cowsay
url: http://bfontaine.net/blog/post/2011/12/10/Cr%C3%A9ez-vos-propres-fichiers-Cowsay
content:
    <p><i>Ce billet est la suite du billet sur <a href="http://bfontaine.net/blog/post/2011/08/30/Afficher-des-citations-dans-la-console-avec-fortune">Fortune et Cowsay</a>.</i></p>
<p>Dans le précédent billet, on a vu comment combiner les commandes <code>fortune</code> et <code>cowsay</code> pour afficher une citation dans une bulle, avec un personnage en ASCII Art (une vache, par défaut). L'utilitaire Cowsay propose pas moins d'une cinquantaine de personnages différents, du squelette au cheval en passant par Dark Vador, Calvin (de <a href="http://www.gocomics.com/calvinandhobbes/" hreflang="en">Calvin &amp; Hobbes</a>) ou encore un <a href="http://fr.wikipedia.org/wiki/Stegosaurus" hreflang="fr">stégosaure</a>. Ce billet vous propose de créer votre propre personnage.</p>
<div>Tout ce dont vous aurez besoin sera votre éditeur de texte préféré, et un peu d'imagination. Pour ce billet, on ne fera pas preuve d'une grande originalité en créant un fichier pour le petit lapin suivant :</div><pre>(\_/)<br />('.')<br />(&gt; &lt;)</pre><p>On crée donc un fichier <code>lapin.cow</code>, dont on commente les premières lignes (à l'aide de <code>#</code>) pour expliquer ce que représente notre dessin :</p>
<pre>#<br /># A little rabbit<br />#</pre>
<p>On précise ensuite que notre dessin personnage va suivre, et qu'on terminera ledit dessin par "<i>EOC</i>" (<i>End Of Cow</i>) :</p>
<pre>$the_cow = &lt;&lt;"EOC";</pre>
<p>On dessine ensuite notre personnage avec quelques backslashs en haut à gauche, pour montrer que la bulle de texte vient de lui. On n'oublie pas de se&nbsp;décaler&nbsp;un peu sur la droite pour que nos backslashs tombent à peu près au milieu de la bulle.</p>
<pre>$the_cow = &lt;&lt;"EOC";<br />    \<br />     \<br />      (\_/)<br />      ('.')<br />      (&gt; &lt;)</pre><p>Vous pouvez bien entendu ajouter des backslashs, surtout si votre personnage est grand. Une fois votre personnage terminé, il faut « échapper » les caractères spéciaux, ici les backslashs du dessin (pas ceux en haut à gauche). Faites précéder chaque backslash du dessin par un second backslash.</p>
<pre>$the_cow = &lt;&lt;"EOC";<br />    \<br />     \<br />      (\\_/)<br />      ('.')<br />      (&gt; &lt;)</pre>
<p>Ensuite, remplacez les backslashs en haut à gauche par <code>$thoughts</code>. Ainsi, le programme <code>cowsay</code> remplacera chaque occurence de <code>$thoughts</code> par un backslash, tandis que <code>cowthink</code> le remplacera par une petite bulle (un « <i>o</i> » minuscule). Il ne reste plus qu'à tenir notre promesse et terminer notre dessin par "<i>EOC</i>", sur une dernière ligne.</p>
<pre>$the_cow = &lt;&lt;"EOC";<br />    $thoughts<br />     $thoughts<br />      (\\_/)<br />      ('.')<br />      (&gt; &lt;)<br />EOC</pre>
<p>Voilà, votre personnage est terminé. Vous pouvez tester le résultat avec l'option -f de cowsay, suivie du chemin vers votre fichier :</p>
<pre>$ cowsay -f ./lapin.cow "Hello World ! "<br />&nbsp;________________<br />&lt; Hello World ! &nbsp;&gt;<br />&nbsp;----------------<br />&nbsp; &nbsp; \<br />&nbsp; &nbsp; &nbsp;\<br />&nbsp; &nbsp; &nbsp; (\_/)<br />&nbsp; &nbsp; &nbsp; ('.')<br />&nbsp; &nbsp; &nbsp; (&gt; &lt;)</pre>
<p>Si vous souhaitez mettre votre personnage avec tous les autres, il suffit de déplacer le fichier au bon endroit (vérifiez avant qu'aucun autre fichier n'a le même nom) :</p>
<pre>$ sudo mv lapin.cow /usr/share/cowsay/cows/</pre><p>Et voilà !</p>


feed_id: urn:md5:9ea393e48cc1aa2327bb15baa36eea44
title: Comment Chromium accélère les téléchargements
uri: Comment-Chromium-accélère-les-téléchargements
url:
http://bfontaine.net/blog/post/2011/12/06/Comment-Chromium-acc%C3%A9l%C3%A8re-les-t%C3%A9l%C3%A9chargements

content:
    <p>J'ai découvert hier soir un comportement assez sympathique de <a href="http://www.chromium.org/Home" hreflang="en">Chromium</a> (qui doit fonctionner aussi sous Chrome), concernant les téléchargements. Lorsque vous faites un clic-droit puis « <i>Enregistrer sous...</i> » pour enregistrer un fichier, une fenêtre s'ouvre pour vous laisser choisir l'emplacement de sauvegarde dudit fichier, jusque là rien d'extraordinaire. Sauf qu'une fois cette fenêtre ouverte, Chromium n'attend pas que vous ayez choisi le nom&nbsp;du fichier pour commencer à le télécharger. Ainsi, il commence déjà à télécharger le fichier sous le doux nom de&nbsp;<code>.org.chromium.Chromium.[id]</code> (<code>id</code> étant un identifiant unique), dans le répertoire de téléchargements. Une fois que vous avez choisi un nom, il n'y a plus qu'à terminer le téléchargement (peut-être est-ce déjà fait) et renommer le fichier.</p>


feed_id: urn:md5:4c1739cfe0b1f4cab177d0c75308689b
title: Exportez votre code Python en HTML avec pyhtmlizer
uri: Exportez-votre-code-Python-en-HTML-avec-pyhtmlizer
url: http://bfontaine.net/blog/post/2011/10/27/Exportez-votre-code-Python-en-HTML-avec-pyhtmlizer
content:
    <p>Il est parfois utile, par exemple dans le cadre d'un blog, de partager des bouts de code, notamment en Python. La commande <code>pyhtmlizer</code> (disponible avec le paquet&nbsp;<code>python-twisted-core</code> sous Ubuntu) permet d'exporter facilement du code Python, avec une coloration syntaxique de notre choix. Elle s'utilise de la façon suivante :</p>
<pre>pyhtmlizer [-s <i>feuille-de-style.css</i>] <i>fichier.py</i></pre>
<p>Un fichier <i>fichier.py.html</i> sera ainsi créé. La commande place tous les mots-clefs de Python dans des balises <code>&lt;span&gt;</code>, avec une classe correspondante au type du mot-clef. Par exemple, toutes les variables auront la classe <code>py-src-variable</code>. Pour les commentaires, ça sera <code>py-src-comment</code>, tandis que pour les instructions, ça sera <code>py-src-keyword</code>. Il ne reste plus qu'à écrire une feuille de style et le tour est joué.</p>
<p>Par exemple, pour le code suivant :</p>
<pre>#! /usr/bin/python3.2<br /><br />def hello(name="Dennis"):<br />    "dit bonjour à &lt;name&gt;"<br />    print("Hello %s !" % name)<br />if (__name__ == "__main__"):<br />    <br />    hello()</pre>
<p>qui ne fait que dire bonjour, et la feuille de style suivante :</p>
<pre>body {background-color:#333;color:#EEE}<br />.py-src-comment {color:#CCC}<br />.py-src-keyword {color:orange;font-weight:bold}<br />.py-src-variable {color:#5AF;font-weight:bold}<br />.py-src-identifier {color:cyan;font-weight:bold}<br />.py-src-string {color:#AFA}<br />.py-src-parameter {color:pink}</pre>
<p>Le résultat est le suivant :</p>
<style>
.py-src-comment {color:#CCC}
.py-src-keyword {color:orange;font-weight:bold}
.py-src-variable {color:#5AF;font-weight:bold}
.py-src-identifier {color:cyan;font-weight:bold}
.py-src-string {color:#AFA}
.py-src-parameter {color:pink}
</style>
<pre style="background-color:#333;color:#EEE;"><span class="py-src-comment">#! /usr/bin/python3.2</span><br /><br /><span class="py-src-keyword">def</span> <span class="py-src-identifier">hello</span>(<span class="py-src-parameter">name=<span class="py-src-string">"Dennis"</span>):<br />&nbsp;&nbsp;&nbsp;&nbsp;<span class="py-src-string">"dit bonjour à &lt;name&gt;"</span><br />&nbsp;&nbsp;&nbsp;&nbsp;<span class="py-src-keyword">print</span>(<span class="py-src-string">"Hello %s !"</span> % <span class="py-src-variable">name</span>)<br /><br /><br /><span class="py-src-keyword">if</span> (<span class="py-src-variable">__name__</span> == <span class="py-src-string">"__main__"</span>):<br />&nbsp;&nbsp;&nbsp;&nbsp;<span class="py-src-variable">hello</span>()</pre>
<p>Bien entendu, vous pouvez modifier votre feuille de style comme vous voulez, et faire des choses très jolies. Il existe des scripts JS qui permettent de colorer automatiquement le code, mais si vous préférez éviter d'avoir 50 scripts par page pour faire de simples effets réalisables en CSS ou HTML, utilisez cette méthode.</p>


feed_id: urn:md5:0385cbd3ea18fe6d66013206bc6b238f
title: R2D2, un an après
uri: R2D2-un-an-après
url: http://bfontaine.net/blog/post/2011/10/07/R2D2%2C-un-an-apr%C3%A8s
content:
    <p>Attention, cet article ne parle pas d'<i>R2-D2</i>, le <a href="http://fr.wikipedia.org/wiki/R2D2">robot de <i>Star Wars</i></a>, mais du réseau social interne à l'Université Paris Diderot (<a href="http://r2d2.script.univ-paris-diderot.fr/"><i>R2D2</i></a>&nbsp;signifie « <i>Réseau, Ressources de Denis Diderot</i> »).</p>
<p>Cela fait bientôt un an que j'ai rejoins <i>R2D2</i>, un réseau social réservé aux étudiants et au personnel de Paris Diderot (Paris VII), qui avait ouvert peu avant. La première chose qui choque, quand on arrive dessus, c'est le tutoiement systématique (comme sur <a href="http://fr.myspace.com/"><i>Myspace</i></a>). La seconde, c'est le trop-plein de fonctionnalités : blogs, télex (équivalents des tweets de <a href="https://twitter.com/" hreflang="en"><i>Twitter</i></a>), pages (équivalents des articles de <a href="https://www.facebook.com/"><i>Facebook</i></a>), favoris, groupes, fichiers, listes d'amis. Il est également possible de suivre des utilisateurs (et inversement). Ceux-ci sont vos « amis », tandis que ceux qui vous suivent sont vos « contacts » (ne cherchez pas la logique, il n'y en a pas). La page d'accueil du réseau permet par ailleurs de voir la « rivière d'activités », une sorte de « <a href="https://twitter.com/public_timeline" hreflang="en">public timeline</a>&nbsp;», c'est-à-dire les activités récentes de tous les membres du réseau, ainsi que les télex récents. Le problème, c'est justement que le réseau est interne à l'Université, donc créer des groupes d'entraide entre étudiants peut être pratique, mais l'utilité de créer un blog avec un nombre de lecteurs aussi limités est discutable. Peut-être aussi parce que le réseau n'a jamais connu de lancement en grandes pompes, il n'y a pas beaucoup d'activité dessus.</p>
<p><i>R2D2</i> est basé sur <a href="http://elgg.org/" hreflang="en"><i>Elgg</i></a>, un moteur de réseau social open source, utilisé par plusieurs autres universités. Ce moteur permet de mettre en place une API, mais les administrateurs d'R2D2 n'en ont pas fait. J'ai donc écrit une <abbr title="Application Programming Interface">API</abbr> en Python, totalement dépendante de la structure des pages (qui n'a pas changée en un an). Elle gère l'authentification <a href="http://fr.wikipedia.org/wiki/Central_Authentication_Service">CAS</a>&nbsp;utilisée sur le site, et permet de faire une recherche selon plusieurs critères, ainsi que de recueillir un certain nombre d'informations publics sur un utilisateur. Elle se veut facile d'utilisation, voici un exemple (l'utilisateur est un faux) :</p>
<pre>&gt;&gt;&gt; import r2d2 as rd
&gt;&gt;&gt; rd.auth("toto", "motdepasse").connection()
&gt;&gt;&gt; u = rd.user("toto", autocomplete=True)
&gt;&gt;&gt; u.prenom
'Toto'
&gt;&gt;&gt; u.nom
'Matte'
&gt;&gt;&gt; {f.pseudo for f in u.friends}
{'Titi', 'Lulu', 'Tata'}</pre>
<p>J'ai utilisé cette API pour générer quelques statistiques. On dénombre ainsi 2173 inscrits, dont 1991 étudiants (soit 5.7% de étudiants inscrits à l'Université, d'après un chiffre de septembre 2010) et 182 profs/membres du personnel. Parmi ces 2173 inscrits, seulement 278 (13%) ont remplacé l'avatar par défaut, et 795 (37%) ont rempli la section "À mon propos". On trouve 1913 utilisateurs (88%) sans amis ni contacts. Parmis le personnel (et les professeurs) celle qui a le plus d'amis (59) est une technicienne du <a href="http://www.script.univ-paris-diderot.fr/">SCRIPT</a>&nbsp;(le service informatique de l'université), tandis que celui qui a le plus de contacts est le professeur <a href="http://www.liafa.jussieu.fr/~yunes/">J.-B. Yunès</a>&nbsp;(18). Côté étudiants, celle qui a le plus d'amis en a 27. Seulement 9 étudiants et 9 membres du personnel ont 10 amis ou plus. Les membres du personnel ont en moyenne 1.54 amis et 0.86 contacts (si on ne compte que ceux qui ont un ami ou plus, cela donne respectivement 6.22 et 4.00). Les étudiants, eux, ont en moyenne 0.32 amis et 0.17 contacts (si on ne compte que ceux qui ont un ami ou plus, cela donne respectivement 2.99 et 2.16). Autant dire que ce n'est pas (encore) un grand succès.</p>


feed_id: urn:md5:67ecb64b5a4ff043321579ac508b5438
title: Contrez la censure avec Autoblog
uri: Contrez-la-censure-avec-Autoblog
url: http://bfontaine.net/blog/post/2011/09/21/Contrez-la-censure-avec-Autoblog
content:
    <p>Le projet <a href="http://sebsauvage.net/streisand.me/fr/index.html">Autoblog</a> est un système permettant de dupliquer très facilement les articles d'un blog à de multiples endroits. Son principe rappelle l'<a href="https://secure.wikimedia.org/wikipedia/en/wiki/Streisand_effect" hreflang="en">effet Streisand</a>&nbsp;: En 2003, Barbara Streisand a déposé une plainte contre un photographe pour éviter la diffusion d'une photographie de sa maison qu'il avait posté sur Internet. Cette plainte a attiré l'attention du public et des médias, et la photo a très vite été dupliquée partout. L'effet Streisand a lieu lorsqu'une tentative de censure a lieu sur le réseau contre une information au départ sans importance. Le réseau se défend de lui-même, pour reprendre les termes de <a href="http://lacantine.ubicast.eu/videos/pses-leffet-flamby/">Benjamin Bayart</a>, et l'information est massivement dupliquée pour contrer ladite censure.</p>
<p>Ici, le principe est très simple : Il suffit de copier deux fichiers sur son site pour dupliquer les articles d'un blog. Un fichier <code>index.php</code>, commun à tous les blogs, et un fichier de configuration, <code>vvb.ini</code> (<i>VVB</i> pour <i>VroumVroumBlog</i>, le programme permettant de faire fonctionner l'autoblog). Le fichier de configuration contient le titre du blog, sa description, son adresse, son flux RSS, et les adresses d'où télécharger les médias. Lorsque la page <code>index.php</code> est visitée, le programme va récupérer le flux RSS du blog (sauf s'il a déjà été mis à jour dans les 60 dernières minutes), et dupliquer les articles sur la page. Ils sont stockés, de même que les médias, dans un répertoire local. Ainsi, si le blog est supprimé, les articles restent sur les autoblogs, et il est toujours possible de les consulter. Le système ne nécessite rien d'autre que PHP5 et ces deux fichiers. Il est possible d'utiliser une <a href="http://blogs.kd2.org/bohwaz/?2011/07/14/369-auto-blog-vroumvroumblog-et-effet-streisand">version améliorée</a> qui utilise SQLite (plus rapide).</p>
<p>Des fichiers de configurations sont disponibles <a href="http://sebsauvage.net/streisand.me/fr/choose.html">ici</a>, celui de ce blog est disponible <a href="http://bfontaine.net/blog/vvb.ini">là</a>, et il très facile d'en <a href="http://sebsauvage.net/streisand.me/fr/vvb.ini.html">créer un</a> à la main. Ils ont la forme suivante:</p>
<pre>[BfontaineBlogConfig]
SITE_TITLE="Autoblog de bfontaine.net"
SITE_DESCRIPTION="Ce site n'est pas le blog officiel de bfontaine.net&lt;br&gt;C'est un blog automatis&amp;eacute; qui r&amp;eacute;plique les articles de &lt;a href="http://bfontaine.net/blog"&gt;bfontaine.net/blog&lt;/a&gt;"
SITE_URL=http://bfontaine.net/blog
FEED_URL=http://bfontaine.net/blog/feed/atom
DOWNLOAD_MEDIA_FROM=bfontaine.net</pre><p>Et cela donne <a href="http://bfontaine.net/autoblog/bfontaine/">cet autoblog</a>&nbsp;(qui n'a aucune autre utilité que de servir d'exemple à ce billet, puisqu'il est situé sur le même hébergement que l'original). J'héberge personnellement <a href="http://bfontaine.net/autoblog/">7 autoblogs</a>, dont celui d'<a href="http://bfontaine.net/autoblog/owni/">OWNI</a>, de <a href="http://bfontaine.net/autoblog/sebsauvage/">Sebsauvage</a>, ou encore de la <a href="http://bfontaine.net/autoblog/quadrature/">Quadrature du Net</a>. Je vous encourage à faire de même, cela prend très peu de place, est très facile à installer, et peut être d'une grande utilité.</p>
<p>À noter que si vous êtes sous Dotclear et que vous voulez que d'autres dupliquent votre blog, pensez à décocher la case "Tronquer les flux de syndicalisation" dans les paramètres du blog. Sans cela, les articles ne seront dupliqués qu'en partie, avec un lien invitant à visiter le blog pour lire la suite...</p>


feed_id: urn:md5:7d0dcf04895e8b4bcd58884488ece19c
title: Utiliser les raccourcis de recherche sous Google Chrome
uri: Utiliser-les-raccourcis-de-recherche-sous-Google-Chrome
url: http://bfontaine.net/blog/post/2011/09/07/Utiliser-les-raccourcis-de-recherche-sous-Google-Chrome
content:
    <p>Une fonctionnalité que j'apprécie sous <a href="http://www.konqueror.org/" hreflang="en">Konqueror</a> est la possibilité d'utiliser de multiples moteurs de recherche directement dans la barre d'adresse. Ainsi, il sufft de préfixer sa recherche par <code>gg:</code> pour faire une recherche sur Google, <code>py:</code> pour une recherche dans la <a href="http://docs.python.org/py3k/" hreflang="en">documentation Python</a>, etc. Bien entendu, ils sont tous modifiables, et on peut en ajouter. Et bien sachez qu'il existe une fonctionnalité similaire sous Google Chrome/Chromium.</p>
<p>Pour cela, il faut faire un clic droit dans la barre d'adresse de votre navigateur, et de cliquer sur "<i>Modifier les moteurs de recherche...</i>". Là, vous tombez sur, vous vous en doutiez, la page de configuration des moteurs de recherche. Celle-ci comporte deux cadres contenant chacun une liste de moteurs de recherche organisés de la façon suivante: à gauche, le nom du moteur, au milieu son mot-clef (et c'est ça qui nous intéresse), et à droite l'URL de recherche formatée. Vous remarquerez ainsi que dans chaque URL, le terme de recherche est remplacé par les deux caractères <code>%s</code>. C'est sur cette page que vous pouvez modifier votre moteur par défaut. Vous pouvez également ajouter un moteur de recherche, par exemple <a href="http://duckduckgo.com/" hreflang="en">DuckduckGo</a>, ou la documentation Python, ou même votre blog préféré. Il suffit d'entrer un terme de recherche aisément reconnaissable dans l'URL. Par exemple avec DuckDuckGo, cherchez <i>tralalalalala</i>, vous obtiendrez une URL de la forme:</p>
<pre>http://duckduckgo.com/?q=tralalalalala</pre><p>Ici, ce n'est pas bien compliqué, il suffit de remplacer&nbsp;<code>tralalalalala</code> par <code>%s</code>&nbsp;lors de l'ajout dans la liste des moteurs de recherche. On se retrouve avec des URL de la forme suivante:</p>
<pre>Documentation Python 3:<br />http://docs.python.org/py3k/search.html?q=%s&amp;check_keywords=yes&amp;area=default<br />Documentation PHP:<br />http://fr2.php.net/manual-lookup.php?pattern=%s&amp;lang=fr<br />Ce blog:<br />http://bfontaine.net/blog/?q=%s<br />etc</pre><p>Associez à chacun de vos moteur de recherche un mot clef simple, par exemple <code>gg</code> pour Google, <code>y!</code> pour Yahoo!, etc. Maintenant, lorsque vous taperez votre mot-clef dans la barre d'adresse, Chrome/Chromium vous proposera d'appuyer sur la touche tabulation afin d'effectuer une recherche sur le moteur de recherche correspondant. Par exemple, pour ma part, taper <code>ddg &lt;tabulation&gt; toto</code> effectuera la recherche <i>toto</i> sur DuckDuckGo.</p>
<h5>Bonus</h5>
<p>Vous pouvez en profiter pour décider d'en dire moins à Google. Ainsi, sous Ubuntu, l'URL de recherche ressemble par défaut à ça:</p>
<pre>http://www.google.fr/search?sourceid=chrome&amp;client=ubuntu&amp;channel=cs&amp;ie=UTF-8&amp;q=ma+recherche</pre>
<p>S'il n'est pas possible de modifier l'URL formatée de Google, il est en revanche possible d'ajouter un "autre" Google (et éventuellement de supprimer le premier), avec l'adresse formatée <code>http://www.google.fr/search?q=%s</code>, ce qui est plus court, plus lisible, et en dit (un tout petit peu) moins à Google.</p>


feed_id: urn:md5:8339929287f75414aa03fdca62e264dd
title: Afficher des citations dans le terminal avec Fortune
uri: Afficher-des-citations-dans-le-terminal-avec-Fortune
url: http://bfontaine.net/blog/post/2011/08/30/Afficher-des-citations-dans-la-console-avec-fortune
content:
    <p><i>Fortune</i> est un utilitaire UNIX utilisable en ligne de commande qui permet d'afficher une citation au hasard, parmi celles de la base de donnée. Il tire son nom des "fortune cookies", ces petits biscuits dans lesquels on trouve un morceau de papier sur lequel on peut lire une prédiction ou une citation humoristique.</p>
<p>Sous Ubuntu, il faut installer le paquet <code>fortune-mod</code> pour des citations en anglais (par défaut), et <code>fortune-fr</code> pour des citations en français. Pour la suite du billet, on&nbsp;considérera&nbsp;que vous avez installé les paquets nécessaires pour les citations en français. Pour l'utiliser, il suffit de taper la commande dans un terminal:</p>
<pre>$ fortune</pre><p>Rien de très compliqué. L'utilitaire pioche une citation au hasard parmi les fichiers dont il dispose. On pourra avoir la liste desdits fichiers, avec, pour chacun, la probabilité qu'une citation provienne du fichier, en ajoutant l'option <code>-f</code>. Plus le fichier est gros, plus la probabilité qu'une citation en provienne est élevée. Si vous avez un fichier bien plus gros que les autres, il peut être intéressant d'ajouter l'option <code>-e</code>, qui donne la même probabilité à chaque fichier. D'autres options existent, par exemple <code>-l</code> qui sélectionne uniquement les citations longues (au contraire de <code>-s</code>),<code> -m &lt;expression&gt;</code> qui permet de ne sélectionner que les citations qui contiennent l'expression régulière donnée, etc. Pour tirer une citation d'un fichier en particulier, il suffit de l'ajouter en argument, exemple:</p>
<pre>$ fortune humour</pre><h5>Ajouter ses propres fichiers</h5>
<p>Il peut être intéressant d'ajouter ses propres fichiers pour diversifier les sources de citations. Les fichiers sont formatés de la façon suivante:</p>
<pre>citation 1<br />%<br />citation 2<br />citation 2 (suite)<br />%<br />citation 3<br />%<br />...</pre><p>Chaque citation est séparée de la suivante par le symbole du pourcentage (<code>%</code>). Il suffit donc d'ouvrir votre éditeur de texte, et de remplir un fichier avec vos citations préférés. Certains sites proposent des fichiers déjà prêts, par exemple les <a href="http://www.chucknorrisfacts.fr/fortunes/fortunes.txt">Chuck Norris Facts</a>&nbsp;(l'encodage pose problème chez moi), <a href="http://danstonchat.com/fortunes">DansTonChat</a>&nbsp;(attention, fichier assez lourd), ou encore des <a href="http://bfontaine.net/fortunes/vimtweets">astuces pour Vi(m)</a> (issues des tweets de <a href="https://twitter.com/#!/vimtips" hreflang="en">@vimtips</a>).</p>
<p>Pour pouvoir ajouter un fichier à <i>fortune</i>, faites les manipulations suivantes (on suppose que vous avez téléchargé un fichier de nom <i>monfichier.txt</i>, qui contient des citations formatés qui parlent de Tintin).</p>
<pre>$ sudo cp ./monfichier.txt /usr/share/games/fortunes/fr/tintin<br />$ strfile monfichier.txt<br />$ sudo mv ./monfichier.txt.dat /usr/share/games/fortunes-fr/tintin.dat<br />$ sudo ln -s /usr/share/games/fortunes-fr/tintin.dat /usr/share/games/fortunes/fr/tintin.dat</pre><p>Et voilà ! Vous pouvez faire un test:</p>
<pre>$ fortune tintin</pre>
<p>Ce qui vous affichera une citation tirée (pseudo-)aléatoirement de votre fichier.</p>
<h5>Bonus: Cowsay</h5>
<p>Avec <i>fortune</i>, les citations s'affichent directement dans le terminal. On peut également avoir un personnage en ASCII Art avec une bulle contenant la citation, comme ci-dessous:</p>
<pre>&nbsp;_____________________________________<br />/ Après une heure de gym, je me sens &nbsp;\<br />| mieux ; Les autres aussi me sentent |<br />| mieux. &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|<br />| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |<br />\ -+- Philippe Geluck, Le chat -+- &nbsp; &nbsp;/<br />&nbsp;-------------------------------------<br />&nbsp; &nbsp; &nbsp; &nbsp; \ &nbsp; ^__^<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;\ &nbsp;(oo)\_______<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; (__)\ &nbsp; &nbsp; &nbsp; )\/\<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ||----w |<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; || &nbsp; &nbsp; ||</pre><p>Pour cela, il faut utiliser l'utilitaire <code>cowsay</code>, qui affiche un personnage (une vache, par défaut) avec une bulle contenant le texte donné en argument (pour avoir un personnage qui pense au lieu de parler, utilisez <code>cowthink</code>. D'autres options de personnalisation sont disponibles, consultez la page de manuel pour plus d'informations). Il est possible de spécifier quel personnage utiliser avec l'option <code>-f</code> suivie du nom du fichier correspondant, les fichiers étant dans <code>/usr/share/cowsay/cows</code>. Pour afficher des citations, on préfèrera utiliser des citations courtes, on utilisera donc la commande suivante:</p>
<pre>$ cowsay "`fortune -se`"</pre><p>Qui affichera une vache disant une citation courte (<code>-s</code>) provenant d'un fichier aléatoire (tous ayant la même probabilité, grâce à l'option <code>-e</code>). Pour utiliser un personnage aléatoire (attention, certains sont très grands), on pourra utiliser la commande suivante:</p>
<pre>$ cowsay -f `\ls /usr/share/cowsay/cows | rl | tail -n 1` "`fortune -se`"</pre><p>On utilise ici un fichier (<code>-f</code>) aléatoire, sélectionné en listant (<code>ls</code>) le répertoire (l'utilisation d'un backslash avant permet d'annuler les éventuels alias), en mélangeant les lignes (<code>rl</code>, obtenu sous Ubuntu en installant le paquet <code>randomize-lines</code>), et en sélectionnant la dernière (<code>tail -n 1</code>, ou &nbsp;plus simplement<code> tail -1</code>).</p>
<p><strong><u>Mise-à-jour 11 déc. 2011 :</u></strong> La création de fichiers pour <code>cowsay</code> a été l'objet d'un billet : <a href="http://bfontaine.net/blog/post/2011/12/10/Cr%C3%A9ez-vos-propres-fichiers-Cowsay">Créez vos propres fichiers pour Cowsay</a>.</p>


feed_id: urn:md5:8b99bce8afaacd45af304bda0c37da3e
title: Réalisez vos diagrammes en ASCII art avec Ditaa
uri: Réalisez-vos-diagrammes-en-ASCII-art-avec-Ditaa
url: http://bfontaine.net/blog/post/2011/08/23/R%C3%A9alisez-vos-diagrammes-en-ASCII-art-avec-Ditaa
content:
    <p><a href="http://sourceforge.net/projects/ditaa/" hreflang="en">Ditaa</a> (<i lang="en">DIagrams Through Ascii Art</i>) est un utilitaire en ligne de commande écrit en Java permettant de transformer un diagramme réalisé en <a href="http://fr.wikipedia.org/wiki/Art_ASCII">ASCII art</a> en image au format png. Il suffit d'ouvrir son éditeur de texte préféré et de créer un diagramme, par exemple le suivant, pour l'exemple:</p>
<pre>+------+      +------+<br />|&nbsp;tutu | -+-&gt; | toto |&lt;-+<br />+------+  |   +------+  |<br />          |             |<br />          |   +------+  |<br />          +-&gt; | lala |--+<br />              +------+</pre>
<p>Puis de lancer Ditaa:</p>
<pre>$ ditaa mondiagramme.txt</pre><p>Et on se retrouve avec un fichier <code>mondiagramme.png</code>, comme le suivant:</p>
<img src="http://bfontaine.net/dotclear2/public/ditaa.png" alt="Diagramme Ditaa" style="margin-top: 0; margin-right: 1em; margin-bottom: 1em; margin-left: 0; " title="Diagramme Ditaa, août 2011" />
<p>Mais Ditaa propose également l'ajout de couleurs, des lignes en pointillés, des listes, des coins arrondis, etc. Plus d'informations sont données sur le <a href="http://ditaa.sourceforge.net/#usage" hreflang="en">site du projet</a>. Le programme en est à sa version 0.9, sortie en juin dernier. Le paquet <code>ditaa</code> est disponible dans les dépôts Debian et Ubuntu, pour les autres, il faut télécharger et décompresser le fichier&nbsp;<a href="http://sourceforge.net/projects/ditaa/files/ditaa/0.9/" hreflang="en">zip du projet</a>.</p>
<h5>Bonus</h5>
<div>Si vous préférez faire vos diagrammes en ASCII art à la souris, utilisez <code>asciio</code>, écrit en Perl et dont la dernière version date de décembre 2008, disponible dans les dépôts Debian et Ubuntu. L'utilitaire est également disponible sous forme d'archive compressée via le site <a href="http://search.cpan.org/dist/App-Asciio/" hreflang="en">CPAN</a>.</div>


feed_id: urn:md5:32f3d62b42ea04e4ebbf1aa800376589
title: Accélérez votre navigation avec Adzhosts
uri: Accélérez-votre-navigation-avec-Adzhosts
url: http://bfontaine.net/blog/post/2011/08/17/Acc%C3%A9l%C3%A9rez-votre-navigation-avec-Adzhosts
content:
    <p>Vous utilisez très probablement <a href="https://addons.mozilla.org/fr/firefox/addon/adblock-plus/">Adblocks</a>&nbsp;pour "bloquer" les publicités sur les sites que vous visitez. C'est très pratique, certes, mais le problème est que cette extension (disponible pour <a href="https://addons.mozilla.org/fr/firefox/addon/adblock-plus/">Firefox</a>, <a href="https://chrome.google.com/webstore/detail/cfhdojbkjhnklbpkdaibdccddilifddb">Chrome/Chromium</a>, <a href="http://adblockplus.org/en/kmeleon" hreflang="en">K-meleon</a>) ne fait que bloquer l'affichage des publicités, mais celles-ci sont quand même chargées par le navigateur. Une solution existe: Adzhosts.</p>
<p>Tout d'abord, petit rappel: Les sites que vous visitez sont sur des serveurs, donc des ordinateurs. Tous les ordinateurs connectés au réseau ont une adresse IP. Afin de ne pas avoir à se souvenir des adresses IP de tous les sites que l'on visite (<i>google.com</i> est quand même plus simple à retenir que <i>209.85.146.105</i>), on a inventé les noms de domaine. Donc les ordinateurs utilisent des IP, les humains des noms de domaine. Pour faire la correspondance, votre navigateur regarde d'abord dans votre fichier <a href="http://fr.wikipedia.org/wiki/Hosts#Localisation" style="font-style: italic; ">hosts</a>, sinon il interroge les serveurs DNS. En gros, votre fichier <i>hosts</i> est organisé de la façon suivante:</p>
<pre>adresse_IP &nbsp; &nbsp;adresse Internet<br />adresse_IP &nbsp; &nbsp;adresse Internet<br />...</pre>
<p>À chaque ligne, il y a une adresse IP suivie d'un(ou plusieurs)&nbsp;nom(s) de domaine&nbsp;correspondant. Si l'adresse que vous avez tapé dans votre navigateur n'est pas dans le fichier <i>hosts</i>, celui-ci vas interroger les serveurs DNS qui lui répondront gentiment.</p>
<p>Il est possible d'éditer manuellement le fichier <i>hosts</i>, et c'est bien ça qui nous intéresse ici. On peut par exemple bloquer un site web en associant son adresse à une adresse IP qui n'existe pas. C'est de cette façon que fonctionne certains logiciels de contrôle parental. Pour bloquer les publicités, il suffit donc de relever les adresses servant à les héberger, et les rediriger vers une adresse qui n'existe pas. Le plus simple est d'utiliser une adresse locale, située entre <i>127.0.0.0</i> et <i>127.255.255.255</i>. C'est là qu'<a href="http://www.adzhosts.fr/">Adzhosts</a> entre en action: il va nous permettre de télécharger une base de données de sites indésirables via Internet, et de mettre à jour le fichier <i>hosts</i> avec celle-ci. Il utilise l'adresse&nbsp;<i>127.254.254.254</i>.</p>
<p>Mais ce fichier hosts peut se transformer en&nbsp;cauchemar&nbsp;si une personne malintentionnée peut y avoir accès. En effet, il suffit de faire correspondre l'adresse IP du site de ladite personne avec une adresse comme <i>mabanque-super-secure.fr</i>, et quand penserez aller sur le site de votre banque, vous serez en réalité sur celui d'un pirate (qui aura pris soin de déguiser son site). Heureusement, il n'y a normalement personne qui a accès au fichier si vous n'avez pas modifié les droits d'accès par défaut. Mais ici, Adzhosts a accès au fichier. J'ai donc trouvé sur Internet (impossible de me rappeler où) une version modifiée qui enlève toutes les lignes qui redirigent vers une adresse différente de&nbsp;<i>127.254.254.254</i>, s'il y en a (normalement non)<i>.</i></p>
<p><i><br /></i></p>
<p>Pour télécharger la dernière version, c'est <a href="http://www.adzhosts.fr/">par ici</a>. Pour la version modifiée (pour Linux uniquement), c'est <a href="http://bfontaine.net/dwnld/adzhosts.sh.zip">par là</a>. Sous Linux, le programme n'est rien d'autre qu'un script shell qui met à jour votre fichier selon la dernière version de la base de données d'Adzhosts. À noter que si vous n'utilisez pas Adblocks, les emplacements normalement réservés à la publicité afficheront un beau <i>404 Not Found</i>, puisque la publicité ne pourra pas être chargée. Adblocks vous permet ainsi de cacher les emplacements des publicités, tandis qu'Adzhosts empêche leur chargement.</p>
<p><strong>Edit du 20/09/2011</strong>:</p>
<p>Attention, il se peut qu'il y ait des faux-positifs dans la liste noire. Par exemple, on y trouve (version actuelle) <a href="http://www.alexa.com/" hreflang="en">www.alexa.com</a> (qui permet d'avoir des statistiques sur les sites les plus visités) et <a href="http://statcounter.com/" hreflang="en">statcounter.com</a>, ainsi que <a href="http://alturl.com/" hreflang="en">alturl.com</a> (raccourcisseur d'URL). La seule solution que j'ai actuellement trouvé consiste à&nbsp;recenser&nbsp;ces faux-positifs et à les supprimer de la liste lors de la mise à jour (j'ai ajouté quelques <code>sed</code> dans la fonction <code>appending()</code> d'<code>adzhost.sh</code>). Bien entendu, j'ai aussi ouvert le fichier <code>/etc/hosts</code> avec les droits root pour supprimer ces entrées.</p>


feed_id: urn:md5:73c4536854c8d27b90f20f608068ed2a
title: Sortie de Clr version 1.4.4c
uri: Sortie-de-Clr-version-1-4-4c
url: http://bfontaine.net/blog/post/2011/08/09/Sortie-de-Clr-version-1.4.4c
content:
    <p>À l'<a href="http://www.univ-paris-diderot.fr/">Université Paris Diderot</a>, chaque étudiant dispose d'un espace de stockage personnel de 100 Mo, pour toutes les années qu'il passera dans l'université. Autant dire que c'est pas beaucoup. De là est venu la nécessité de nettoyer très régulièrement mon espace de stockage afin de ne pas perdre un octet de place disponible. J'ai donc développé dès la première année un petit script shell sans prétention, intitulé sobrement <i>clr.sh</i> (pour "clear", mais ce mot est déjà pris par la commande du même nom qui permet d'<a href="http://fr.wikipedia.org/wiki/Clear_(Unix)">effacer l'écran du terminal</a>). Au tout début, il ne faisait qu'effacer les fichiers de sauvegarde générés par XEmacs, terminés par un <i>tilde</i> (le caractère '<i>~</i>'). Puis ont été rajoutés les fichiers générés lors de la compilation des documents TeX.</p>
<p>J'ai récemment adapté cet outil pratique à Ubuntu, et la version 1.4.4c est disponible sur Launchpad <a href="https://launchpad.net/clr/+download" hreflang="en">par ici</a>. Un manuel d'installation est disponible (en anglais) <a href="https://answers.launchpad.net/clr/+faq/1732" hreflang="en">par là</a>. Cette version supporte une trentaine d'applications, et n'a été testée que sur Ubuntu, bien qu'elle soit très probablement compatible avec les autres distributions de GNU/Linux et *BSD (l'outil originel a été développé sous FreeBSD).</p>


feed_id: urn:md5:3ccf7ecad3e7838d0d62f8aa732b8ff6
title: Mot de passe oublié ?
uri: Mot-de-passe-oublié
url: http://bfontaine.net/blog/post/2011/07/03/Mot-de-passe-oubli%C3%A9
content:
    <p>La quasi-totalité des sites proposant demandant une identification&nbsp;proposent une procédure pour les étourdis&nbsp;(<a href="http://www.quora.com/" hreflang="en">Quora</a>&nbsp;n'en propose pas, par exemple). Il en existe plusieurs sortes, que l'on peut lister ici:</p>
<h5>La classique</h5>
<p>La procédure classique consiste à renvoyer le (nouveau) mot de passe par e-mail. La méthode la moins sécurisée envoie le mot de passe actuel par e-mail, ce qui implique que les mots de passe sont stockés <u>en clair</u> dans la base de données. Grave erreur ! Si un pirate arrivait à récupérer tout ou une partie de cette base de données, c'en est fini pour vous. Une méthode plus sécurisée consiste à chiffrer les mots de passe dans la base de données (<a href="http://fr.wikipedia.org/wiki/MD5">md5</a>,<a href="http://fr.wikipedia.org/wiki/SHA-1"> sha1</a>, etc), et à en re-générer un qui sera envoyé par e-mail.</p>
<p>Le problème de cette procédure, c'est que si la boîte mail de l'utilisateur a été piratée ou simplement si quelqu'un d'autre y a accès, la personne tier pourra voir le nouveau mot de passe.</p>
<h5>L'originale</h5>
<p>Une procédure plus originale consiste à envoyer le nouveau mot de passe par sms. La plupart des opérateurs mobiles le permettent, ainsi que des sites comme <a href="http://support.ankama.com/fr/faq/1319-comment-reinitialiser-mon-mot-passe-sms">Ankama</a>&nbsp;ou <a href="http://www.multiwizz.com/authentification/sendpassword">Multiwizz</a>. Elle a ses inconvénients: l'utilisateur doit accepter de fournir son numéro de téléphone portable, et l'envoi du sms a un coût. Il est possible d'envoyer des sms via l'API de certains hébergeurs (<a href="http://www.ovh.com/fr/sms_et_fax/pack_sms_fiche_technique.xml">OVH</a>&nbsp;le permet, tandis que&nbsp;<a href="http://faq.1and1.fr/applications/sms/1.html">1&amp;1</a>&nbsp;propose une interface d'administration mais pas d'API). Il existe également des offres extérieures (<a href="http://www.tm4b.com/" hreflang="en">TM4B</a>, par exemple). Les tarifs varient entre 0.08€ et 0.12€ généralement, mais peuvent atteindre 0.20€ (HT) en fonction du prestataire et de la quantité que vous commandez.</p>
<p>L'avantage de cette procédure, c'est qu'elle est plus sécurisée que la précédente, mais elle est payante.</p>
<h5>La sécurisée</h5>
<p>La procédure la plus sécurisée est l'envoi d'un courrier, mais elle est également la plus longue et la plus coûteuse. Elle est à réserver pour des sites web où la sécurité doit être élevée (sites de banques, par exemple). Free l'utilise en cas d'oubli de mot de passe pour les <a href="http://pagesperso.free.fr/">pages persos</a>.</p>
<h5>L'intégrée</h5>
<p>La procédure intégrée permet de réinitialiser son mot de passe directement sur le site concernée. C'est la plus simple pour l'utilisateur, mais le plus dur est de s'assurer que c'est bien le propriétaire du compte qui fait la demande de réinitialisation. Il est possible, par exemple, de demander la réponse à une ou plusieurs questions secrètes qui auront été définies à l'inscription. On peut également poser des questions sur des champs cachés du profil, que seul l'auteur connaît. Lorsque l'on tente de se connecter à Facebook depuis un autre pays, celui-ci effectue une vérification d'identité via les photos des amis du compte; il présente une photo avec trois possibilités, et demande de choisir le nom de celui ou celle qui est représenté(e) sur la photo. Certains sites de MMORPG proposent d'envoyer un scan d'une carte d'identité pour confirmer que l'on est bien le propriétaire du compte. Cette dernière méthode est lourde à gérer autant du côté utilisateur qu'administrateur.</p>
<p>Si l'on est sûr de pouvoir identifier le propriétaire d'un compte de façon automatique sans se tromper, cette procédure est la plus intéressante.</p>
<h5>La personnalisée</h5>
<p>La procédure personnalisée consiste à contacter un responsable du site. Elle se rapproche de la précédente, car la seule tâche du responsable sera de déterminer si la personne qui l'a contacté peut prouver qu'elle est bien propriétaire du compte. Cette procédure reste extrêmement rare, puisqu'elle demande des moyens humains adaptés à l'audience du site, et peut être remplacée par l'une des précédentes.</p>
<hr>
<p>À mon sens, la meilleure procédure est l'<i>intégrée</i>, ou alors un mélange de plusieurs procédures, comme l'e-mail et le sms, avec un message à l'administrateur si vraiment cela ne marche pas. Mais aucune des procédure n'est adaptée à 100% des cas.</p>


feed_id: urn:md5:2da26a8e040430585434efde37ed63bc
title: [Tuto] Un bouton Twitter animé 100% CSS3
uri: Tuto-Un-bouton-Twitter-animé-100-CSS3
url: http://bfontaine.net/blog/post/2011/06/27/Un-bouton-Twitter-anim%C3%A9-100-CSS3
content:
    <p>Aujourd'hui, je vous propose de réaliser un bouton Twitter animé en CSS3, similaire à celui présent en bas à gauche de ma <a href="http://bfontaine.net/">page d'accueil</a>.</p>
<p>On aura besoin pour cela:</p>
<ul><li>D'un éditeur de texte pour le code</li>
<li>De télécharger la police <a href="http://bfontaine.net/styles/fonts/picowa.TTF"><i>Picowa</i></a>, qu'on utilisera pour le logo.</li>
</ul>
<p>On commence par écrire la structure en HTML; On utilisera un <code>&lt;div&gt;</code>&nbsp;contenant deux paragraphes:</p>
<pre>&lt;div id="twitter"&gt;
&nbsp;&nbsp;&nbsp;&nbsp;&lt;p class="logo"&gt;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;a href="http://twitter.com/#!/pseudo" title="pseudo sur Twitter"&gt;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;abbr title="Twitter"&gt;t&lt;/abbr&gt;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/a&gt;
&nbsp;&nbsp;&nbsp;&nbsp;&lt;/p&gt;
&nbsp;&nbsp;&nbsp;&nbsp;&lt;p class="name"&gt;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;a href="http://twitter.com/#!/pseudo" title="pseudo sur Twitter"&gt;@pseudo&lt;/a&gt;
&nbsp;&nbsp;&nbsp;&nbsp;&lt;/p&gt;
&lt;/div&gt;</pre>
<p>La présence d'une balise <code>&lt;abbr&gt;</code> permet de rendre ce "<i>t</i>" plus compréhensible. J'utilise la balise <code>&lt;p&gt;</code> car je veux désigner deux (petits) blocs de texte, c'est donc la balise la plus appropriée. J'utilise les classes <i>logo</i> et <i>name</i> pour pouvoir adapter ce type de bouton à d'autres applications. Ainsi, il me suffira d'utiliser la même structure avec ces classes-là pour obtenir le même comportement.</p>
<p>En haut de notre feuille de style (ou tout au moins avant la stylisation de notre bouton), on importe la police <i>Picowa</i> avec <code>@font-face</code>, qui fonctionne de la façon suivante:</p>
<pre>@font-face {
&nbsp;&nbsp;&nbsp;&nbsp;font-family: "Le Nom que je veux utiliser";
&nbsp;&nbsp;&nbsp;&nbsp;src: url('le chemin vers ma police/picowa.TTF');
}</pre>
<p>Ici, on a donc:</p>
<pre>@font-face {
&nbsp;&nbsp;&nbsp;&nbsp;font-family: "Picowa";
&nbsp;&nbsp;&nbsp;&nbsp;src: url('.../picowa.TTF');
}</pre>
<p>On ajoute également une précision pour annuler un comportement de Firefox qui consiste à ajouter une bordure en pointillés en dessous des abréviations:</p>
<pre>abbr[title] {border-bottom:none}</pre>
<p>On donne une taille à notre bouton, qui sera de 40x40 pixels:</p>
<pre>#twitter {
&nbsp;&nbsp;&nbsp;&nbsp;height:40px;
&nbsp;&nbsp;&nbsp;&nbsp;width:40px;
&nbsp;&nbsp;&nbsp;&nbsp;overflow:visible;
&nbsp;&nbsp;&nbsp;&nbsp;padding:0px;
&nbsp;&nbsp;&nbsp;&nbsp;pdisplay:inline-block;
&nbsp;&nbsp;&nbsp;&nbsp;margin:5px;
}</pre>
<p>On normalise les liens, et on passe les paragraphes en position <code>relative</code>:</p>
<pre>#twitter a,
#twitter a:visited {
&nbsp;&nbsp;&nbsp;&nbsp;color:inherit;
&nbsp;&nbsp;&nbsp;&nbsp;text-decoration:inherit;
}
#twitter &gt; .logo a {
&nbsp;&nbsp;&nbsp;&nbsp;cursor:inherit;
}
#twitter &gt; p {
&nbsp;&nbsp;&nbsp;&nbsp;position:relative;
}</pre>
<p>On s'attaque au logo:</p>
<p>Il sera blanc sur fond bleu ciel, de taille 2em, centré, avec une bordure arrondie, et, surtout, en police <i>Picowa</i>:</p>
<pre>#twitter &gt; .logo {
&nbsp;&nbsp;&nbsp;&nbsp;color:white;
&nbsp;&nbsp;&nbsp;&nbsp;font-size:2em;
&nbsp;&nbsp;&nbsp;&nbsp;font-family:"Picowa", verdana;
&nbsp;&nbsp;&nbsp;&nbsp;background-color:#abcdef;
&nbsp;&nbsp;&nbsp;&nbsp;padding:4px;
&nbsp;&nbsp;&nbsp;&nbsp;margin:0px;
&nbsp;&nbsp;&nbsp;&nbsp;height:100%;
&nbsp;&nbsp;&nbsp;&nbsp;width:38px;
&nbsp;&nbsp;&nbsp;&nbsp;text-align:center;
&nbsp;&nbsp;&nbsp;&nbsp;vertical-align:middle;
&nbsp;&nbsp;&nbsp;&nbsp;border-radius:6px;</pre>
<p>J'utilise la police <i>Verdana</i> au cas où la propriété <code>@font-face</code> ne serait pas supportée. Je précise également un <code>z-index</code> de 2, de façon à pouvoir cacher mon nom derrière, qui sortira sur la droite lorsqu'on passera la souris sur le logo. Enfin, je souhaite que le curseur ne trahisse pas la présence du lien (qui n'est là que pour les interfaces tactiles), et de l'ombre portée:</p>
<pre>&nbsp;&nbsp;&nbsp;&nbsp;z-index:2;
&nbsp;&nbsp;&nbsp;&nbsp;cursor:default;
&nbsp;&nbsp;&nbsp;&nbsp;box-shadow:5px 5px 10px #bcdeff inset;
}</pre>
<p>Par soucis de lisibilité, je n'écris pas les propriétés avec les préfixes de navigateur. Notre bouton a maintenant l'apparence suivante:</p>
<img src="http://bfontaine.net/dotclear2/public/button-t.png" alt="Bouton twitter" style="margin-top: 0; margin-right: auto; margin-bottom: 0; margin-left: auto; display: block; " />
<p>On s'attaque ensuite à l'affichage du pseudo: J'utilise la police <i>Georgia</i>, sur un fond jaune clair, en gras, avec le texte aligné à droite:</p>
<pre>#twitter &gt; .name {
&nbsp;&nbsp;&nbsp;&nbsp;font-family: georgia, serif;
&nbsp;&nbsp;&nbsp;&nbsp;background-color:#ffd;
&nbsp;&nbsp;&nbsp;&nbsp;font-weight:bold;
&nbsp;&nbsp;&nbsp;&nbsp;text-align:right;</pre>
<p>Ici, je souhaite que, par défaut, mon pseudo reste caché, donc je précise un <code>z-index</code> à 1, et des dimensions adaptées. De plus, je souhaite qu'ici, le curseur montre bien la présence d'un lien:</p>
<pre>&nbsp;&nbsp;&nbsp;&nbsp;margin:0px 5px 0px 0px;
&nbsp;&nbsp;&nbsp;&nbsp;padding:6px 5px 4px 4px;
&nbsp;&nbsp;&nbsp;&nbsp;top:-42px;
&nbsp;&nbsp;&nbsp;&nbsp;height:24px;
&nbsp;&nbsp;&nbsp;&nbsp;left:15px;
&nbsp;&nbsp;&nbsp;&nbsp;width:10px;
&nbsp;&nbsp;&nbsp;&nbsp;overflow:hidden;
&nbsp;&nbsp;&nbsp;&nbsp;z-index:1;
&nbsp;&nbsp;&nbsp;&nbsp;cursor:pointer;</pre>
<p>Enfin, j'ajoute des bordures arrondies et un dégradé gris sur les bords:</p>
<pre>&nbsp;&nbsp;&nbsp;&nbsp;border-top:2px solid lightgray;
&nbsp;&nbsp;&nbsp;&nbsp;border-right:2px solid lightgray;
&nbsp;&nbsp;&nbsp;&nbsp;border-bottom:2px solid lightgray;
&nbsp;&nbsp;&nbsp;&nbsp;border-top-right-radius:4px;
&nbsp;&nbsp;&nbsp;&nbsp;border-bottom-right-radius:4px;
&nbsp;&nbsp;&nbsp;&nbsp;box-shadow:0px 0px 2px #aaa inset;
}</pre>
<p>Visuellement, rien n'a changé puisque le second bloc est caché sous le premier. On va faire en sorte qu'on le voit lorsque l'on passe la souris dessus:</p>
<pre>#twitter:hover &gt; .name {
&nbsp;&nbsp;&nbsp;&nbsp;width:88px;
&nbsp;&nbsp;&nbsp;&nbsp;left:33px;
}</pre>
<p>Ces deux valeurs sont à modifier en fonction de la longueur de votre pseudo. Maintenant, lorsque l'on passe la souris sur le logo, voici ce qu'on peut voir:</p>
<img src="http://bfontaine.net/dotclear2/public/button-t2.png" alt="Bouton Twitter avec pseudo" style="margin-top: 0; margin-right: auto; margin-bottom: 0; margin-left: auto; display: block;" />
<p>Afin de rendre notre bouton plus agréable, on va y ajouter des animations: lorsque l'on passera la souris sur le logo, celui-ci pivotera de 90° <em>puis</em> le pseudo sortira:</p>
<pre>#twitter:hover &gt; .logo {
&nbsp;&nbsp;&nbsp;&nbsp;transform:rotate(-90deg);
}</pre>
<p>Soyons précis: lorsque je passe la souris sur le logo, celui-ci pivote de 90° en 0.2 secondes, puis (soit 0.2 secondes après le passage de la souris) le pseudo sort en 0.2 secondes. Ensuite, lorsque je retire la souris, le pseudo se cache en 0.2 secondes, puis (soit 0.2 secondes après que j'eû retiré la souris) le logo retourne à sa position normale en 0.2 secondes. Ceci me permet de regrouper les propriétés CSS.</p>
<p>D'une part, les transitions immédiates:</p>
<pre>#twitter:hover &gt; .logo,
#twitter &gt; .name {
&nbsp;&nbsp;&nbsp;&nbsp;transition:all 0.2s ease-in-out;
}</pre>
<p>De l'autre, celles avec un délai de 0.2 secondes:</p>
<pre>#twitter &gt; .logo,
#twitter:hover &gt; .name {
&nbsp;&nbsp;&nbsp;&nbsp;transition:all 0.2s 0.2s ease-in-out;
}</pre>
<p>Voilà <a href="http://bfontaine.net/exemples/bouton-twitter/">le résultat</a> avec les préfixes de navigateurs:</p>
<img src="http://bfontaine.net/dotclear2/public/.button-t3_m.jpg" alt="Bouton Twitter: évolution" style="margin-top: 0; margin-right: auto; margin-bottom: 0; margin-left: auto; display: block;" /><p>N'hésitez pas à copiez tout ou une partie de la <a href="http://bfontaine.net/exemples/bouton-twitter/twitter_ex_css3_build.css">feuille de style</a>, et à adapter cet exemple à vos goûts.</p>


feed_id: urn:md5:237e73b2b4c892b1304ebe7e202cd1aa
title: iTunes: Destination cadeaux
uri: iTunes-Destination-cadeaux
url: http://bfontaine.net/blog/post/2011/06/23/iTunes%3A-Destination-cadeaux
content:
    <p>Chaque semaine, <a href="http://itunes.apple.com/WebObjects/MZStore.woa/wa/viewMultiRoom?fcId=431369628">iTunes offre un single gratuitement</a>, disponible au téléchargement du mardi au mardi de la semaine suivante. Cette semaine, pas de single, mais une sélection de cadeaux pour l'opération "<a href="http://itunes.apple.com/WebObjects/MZStore.woa/wa/viewMultiRoom?fcId=443177465">Destination cadeaux</a>". Pour fêter l'arrivée de l'été, iTunes offre donc un album, <i>Summer Hits</i>, qui, comme son nom ne l'indique pas, comprend 4 morceaux sans grand intérêt (Christophe Maé, Shy'm, Judith (qui ?), Grégoire), et un clip de James Blunt. Côté application, <i>Paris Match</i>, <i>Jack et le Haricot magique</i> (pour les enfants) et le jeu <i>Babel Rising</i> (Dans lequel vous incarnez un Dieu qui tente d'empêcher des humains de construire une tour de Babel). Des eBooks sont également offerts, un <a href="http://boutique.petitfute.com/italie-2011.html?SID=otrec2n4m9jsm3v0pdmim8qq24">guide de l'Italie</a> (Édité chez <i>Petit Futé</i>), un livre pour enfants (<u>Histoires pour tous les soirs</u>) et un roman, <u>Légende</u>, de David Gemmel. Les eBooks sont uniquement lisibles via l'application <i>iBooks</i>, sur iPad, iPhone et iPod Touch, comme les applications. Inutile de les télécharger si vous ne possédez aucun de ces appareils. Enfin, deux épisodes de séries, le pilote de <i>The Mentalist</i>, et le premier épisode de <i>Vampire Diaries</i>, tous deux uniquement en version française (pas de VOST).</p>
<p>Pour profiter de ces cadeaux, c'est par ici: <a href="http://itunes.apple.com/WebObjects/MZStore.woa/wa/viewMultiRoom?fcId=443177465">Destination cadeaux</a>.</p>


feed_id: http://blog.bfontaine.net/2015/01/17/preventing-bash-pranks
title: Preventing Bash Pranks
uri: Preventing-Bash-Pranks
url: http://blog.bfontaine.net/2015/01/17/preventing-bash-pranks/
content:
<p>The easiest and most popular Bash pranks involve someone messing up with your
<code>~/.bashrc</code>. For example, here is a real-life example:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
</pre></td><td class='code'><pre><code class='bash'><span class='line'><span class="c">#! /bin/bash</span>
</span><span class='line'><span class="nv">b</span><span class="o">=</span>~/.bashrc
</span><span class='line'><span class="nb">echo</span> &gt;&gt;<span class="nv">$b</span>
</span><span class='line'><span class="nb">echo</span> <span class="s2">&quot;echo sleep 1 &gt;&gt;$b&quot;</span> &gt;&gt;<span class="nv">$b</span>
</span></code></pre></td></tr></table></div></figure>


<p>If you execute this script, it’ll add a newline in your <code>~/.bashrc</code> just in
case it doesn’t end with a newline, then add this line:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='bash'><span class='line'><span class="nb">echo </span>sleep 1 &gt;&gt;~/.bashrc
</span></code></pre></td></tr></table></div></figure>


<p>The effect of this isn’t immediately visible to the pranked user. When they’ll
start a new Bash session, e.g. by opening a new terminal window, the code in
<code>~/.bashrc</code> will be executed, and the previous line will add <code>sleep 1</code> at the
end of it, which means it’ll be executed and the user will have to wait one
more second before having their prompt. The next time they’ll open a session,
it’ll add one more line and thus will wait 2 seconds, and so forth.</p>

<p>In this post, I’ll give you an overview of the existing solutions to prevent
these pranks.</p>

<!--more-->


<p>Note that I’m referring to <code>~/.bashrc</code> as your startup Bash file because it’s
commonly used, but some people directly use <code>~/.bash_profile</code> instead, or
another one. When you start a session, Bash reads <code>/etc/profile</code>, then tries
<code>~/.bash_profile</code>, <code>~/.bash_login</code>, and <code>~/.profile</code>, (in that order). In most
environments the default <code>~/.bash_profile</code> file sources <code>~/.bashrc</code>.</p>

<h2>User Rights</h2>

<p>The first solution is to protect your <code>~/.bashrc</code> by restraining the access.
Nobody should be able to edit your file except <em>you</em> (and <code>root</code>). It should be
the default, but if you messed up with user rights, here is how to reset the
file to a safe state (read and write for you, and that’s <em>all</em>):</p>

<pre><code>$ chmod 600 ~/.bashrc
</code></pre>

<p>Most attacks thus involve you executing a script, which allows them to bypass
the rights because the script is executed by <em>you</em> with <em>your</em> editing rights.</p>

<p>One solution would be to remove your own writing right and adding it only when
you need it:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
</pre></td><td class='code'><pre><code class='bash'><span class='line'><span class="c"># add this in your ~/.bashrc</span>
</span><span class='line'>secure-edit<span class="o">()</span> <span class="o">{</span> chmod u+w <span class="nv">$*</span>; <span class="k">${</span><span class="nv">EDITOR</span><span class="k">:-</span><span class="nv">vi</span><span class="k">}</span> <span class="nv">$*</span>; chmod u-w <span class="nv">$*</span>; <span class="o">}</span>
</span></code></pre></td></tr></table></div></figure>


<p>Then remove your writing right:</p>

<pre><code>$ chmod 400 ~/.bashrc
</code></pre>

<p>You can’t edit your file anymore, but you can use your new <code>secure-edit</code>
command:</p>

<pre><code>$ secure-edit ~/.bashrc
</code></pre>

<p>It temporarily allows you to modify the file, open your editor, then put the
restricted rights back.</p>

<h2>The “last line protection”</h2>

<p>This one is easy to use but easy to circumvent. The goal is to prevent one-line
insertions, such as:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='bash'><span class='line'><span class="nb">echo</span> <span class="s2">&quot;alias ls=cd&quot;</span> &gt;&gt; ~/.bashrc
</span></code></pre></td></tr></table></div></figure>


<p>and the solution is as simple as:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='bash'><span class='line'><span class="c">#</span>
</span></code></pre></td></tr></table></div></figure>


<p>Yes, that’s just an hash symbol. If you ends your <code>~/.bashrc</code> with it, the
first inserted line will be commented out:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='bash'><span class='line'><span class="c">#alias ls=cd</span>
</span></code></pre></td></tr></table></div></figure>


<p>It doesn’t work if the prankster adds multiple lines, or adds a newline before
the prank.</p>

<h2><code>return</code></h2>

<p>You can exit from a Bash script with <code>exit</code>. Your <code>~/.bashrc</code> is not executed
like a script, it’s <em>sourced</em>. This means Bash doesn’t start a subshell for it
and execute all its content in the current shell. This also means if you write
<code>exit</code> it’ll exit your current shell.</p>

<p>The solution here is to use <code>return</code> at the end of your file:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='bash'><span class='line'><span class="k">return</span>
</span></code></pre></td></tr></table></div></figure>


<p>Any line added after this one won’t be executed because Bash will stop the
evaluation. Note that while it’s better than the previous solution, it can be
nullified by a <code>sed</code> call (e.g. <code>sed 's/return//'</code>).</p>

<h2>The disguised <code>return</code></h2>

<p>This one is the same as the previous one, but prevents pranksters from removing
it with calls to <code>sed</code> or similar search &amp; replace techniques. It uses the fact
than in Bash you can execute a command contained in a variable by using it at
the proper place:</p>

<pre><code>print_something=echo
$print_something hello
</code></pre>

<p>These lines are equivalent to <code>echo hello</code>. We use the same thing here with
<code>return</code>. The idea is to execute an obfuscated version of return, e.g.:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
</pre></td><td class='code'><pre><code class='bash'><span class='line'><span class="nv">b</span><span class="o">=</span>tu
</span><span class='line'><span class="nv">a</span><span class="o">=</span>re
</span><span class='line'><span class="nv">c</span><span class="o">=</span>rn
</span><span class='line'>
</span><span class='line'><span class="nv">$a$b$c</span>
</span></code></pre></td></tr></table></div></figure>


<p>And <em>voilà!</em> It’s now nearly impossible to detect the <code>return</code> execution
without manually editing the <code>~/.bashrc</code> file.</p>

<p>This is still vulnerable to file replacement, e.g.:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
</pre></td><td class='code'><pre><code class='bash'><span class='line'>rm ~/.bashrc
</span><span class='line'><span class="nb">echo</span> <span class="s1">&#39;echo &quot;sorry, no.&quot;&#39;</span> &gt; ~/.bashrc
</span></code></pre></td></tr></table></div></figure>


<p>This wipes the existing <code>~/.bashrc</code> file and replace it with another one.</p>


feed_id: http://blog.bfontaine.net/2014/11/11/a-python-toolbox
title: A Python toolbox
uri: A-Python-toolbox
url: http://blog.bfontaine.net/2014/11/11/a-python-toolbox/
content:
<p>I started learning Python four years ago and have been heavily programming with
it for near than a year now. In this post I’ll share some tools I use to ease
and speed-up my workflow, either in the Python code or in the development
environment.</p>

<!-- more -->


<p>These tips should be OS- and editor-independent. I have some useful Vim plugins
to work with Python, but that’ll be for another post. You might have to adapt
commands if you work on Windows.</p>

<h2>Setting Things Up</h2>

<p>Let’s say you’re starting a Python library. You have a couple dependencies, and
hopefully you’d like it to work on multiple versions, like 2.6, 2.7, and 3.x.
How can you test that? You have to (a) manage your dependencies to not mess up
with your user environment, and (b) test with multiple Python versions.</p>

<p>Introducing <a href="http://virtualenv.readthedocs.org/en/latest/virtualenv.html">virtualenv</a>.</p>

<p>Virtualenv lets you create <em>isolated</em> Python environments. That means you get a
pristine environment where you’ll install your library’s dependencies, and
nothing else. It’ll be independent of your user space. It’s important to work
with an isolated environment because you don’t know which environment will your
users have, so you shouldn’t make any assumption besides your own requirements.
Working with your user environment means you might forgot a dependency because
it just works since it’s installed on your computer but it’ll broke if run on
another computer without this dependency.</p>

<p>You should be able to install it with <code>pip</code>:</p>

<pre><code>$ pip install virtualenv
</code></pre>

<p>It needs a directory to store the environment. I usually use <code>venv</code>, but you
can choose whatever you want:</p>

<pre><code>$ virtualenv venv
</code></pre>

<p>You can then either “activate” the environment, which adds its <code>bin</code> directory
in your <code>PATH</code>:</p>

<pre><code>$ source venv/bin/activate
$ python  # that's your virtualenv's Python
$ deactivate
$ python  # that's your Python
</code></pre>

<p>or prefix your commands with <code>venv/bin/</code> (replace <code>venv</code> with your directory’s
name):</p>

<pre><code>$ venv/bin/python  # that's your virtualenv's Python
$ python  # that's your Python
</code></pre>

<p>I usually do the later. Install dependencies in the environment:</p>

<pre><code>$ venv/bin/pip install your-dependency
</code></pre>

<p>Don’t forget to tell Git or any tool you’re using for versioning to ignore
this <code>venv</code> directory. It can take some place (from 12MB to more than 40MB)
depending on the number of dependencies you’re relying on.</p>

<p>To remove an environment, just delete its directory:</p>

<pre><code>$ rm -rf venv
</code></pre>

<p>It can be convenient to save place on your computer if you have dozen of
environments for different projects, especially if can quickly re-create any
environment with its dependencies.</p>

<p>If you’re using <code>pip</code> to manage them, you should know you can install
dependencies not only from the command line, but also from a file, with one
dependency per line. Each one of them is processed as if it were given on the
command-line.</p>

<p>For example, you could have <a href="https://github.com/bfontaine/term2048/blob/master/requirements.txt">a file</a> containing this:</p>

<pre><code>colorama==0.2.7
coverage==3.7.1
pep8==1.4.6
py==1.4.20
tox==1.7.0
argparse&gt;=1.1
virtualenv==1.11.4
</code></pre>

<p>This file is usually called <code>requirements.txt</code>, but here again you can call it
whatever you want. You call install all these at once with this command:</p>

<pre><code>$ pip install -r requirements.txt
</code></pre>

<p>But we’re programmers and we’re lazy, we don’t want to track down each
installed library to include it in this file.</p>

<p>Here comes <code>pip freeze</code>.</p>

<p><code>pip freeze</code> outputs all installed libraries with their version. It can be put
in our <code>requirements.txt</code> for later use:</p>

<pre><code>$ pip freeze &gt; requirements.txt
</code></pre>

<p>This <code>requirements.txt</code> file becomes handy when used with <code>virtualenv</code> because
you’re now able to fire up a new environment and install all required
libraries, with two commands:</p>

<pre><code>$ virtualenv venv
$ venv/bin/pip install -r requirements.txt
</code></pre>

<p>Note that these are libraries used in the environment, not necessarily your
library’s dependencies. In the above example you can see we’re installing
<code>coverage</code> and <code>pep8</code>, which respectively are a code coverage test tool and a
lint one we’ll talk about later in this post, not libraries we’re depending on
here.</p>

<p>You should thus add this file in your public repository, because it provides
anyone with the informations they need to have in order to mirror your
environment and be able to contribute to your project.</p>

<h2>Coding</h2>

<p>Now that your local environment is ready, you can start coding your library.
You’ll often have to fire an interpreter to test some things, use <code>help()</code> to
check a function’s arguments, etc. Having to type the same things over and over
takes time, and remember: we’re lazy.</p>

<p>Like Bash and some other tools, the Python interpreter can be configured with
an user file, namely <code>$PYTHONSTARTUP</code>. It allows you to add autocompletion,
import common modules, and execute pretty much any code you want.</p>

<p>Start by setting <code>PYTHONSTARTUP</code> in your <code>~/.bashrc</code> (if you’re using Bash):</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='bash'><span class='line'><span class="nb">export </span><span class="nv">PYTHONSTARTUP</span><span class="o">=</span><span class="s2">&quot;$HOME/.pythonrc.py&quot;</span>
</span></code></pre></td></tr></table></div></figure>


<p>Here we’re telling the interpreter to look for the <code>$HOME/.pythonrc.py</code> file
and executing it before giving us a prompt.</p>

<p>Let’s initialize this file with autocompletion support:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="k">try</span><span class="p">:</span>
</span><span class='line'>    <span class="kn">import</span> <span class="nn">readline</span>
</span><span class='line'><span class="k">except</span> <span class="ne">ImportError</span><span class="p">:</span>
</span><span class='line'>    <span class="k">pass</span>
</span><span class='line'><span class="k">else</span><span class="p">:</span>
</span><span class='line'>    <span class="kn">import</span> <span class="nn">rlcompleter</span>
</span><span class='line'>    <span class="n">readline</span><span class="o">.</span><span class="n">parse_and_bind</span><span class="p">(</span><span class="s">&quot;tab: complete&quot;</span><span class="p">)</span>
</span></code></pre></td></tr></table></div></figure>


<p>You can add a lot more <a href="http://www.sontek.net/blog/2010/12/28/tips_and_tricks_for_the_python_interpreter.html">stuff</a> in this file, like <a href="https://github.com/bfontaine/Dotfiles/blob/master/.pythonrc.py#L21-L38">history</a>
support, <a href="https://github.com/coderholic/config/blob/master/.pythonrc.py#L77-L80">colored prompts</a> or common imports. For example, if you use <code>sys</code>
and <code>re</code> a lot, you can save time by adding them in your startup file:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="kn">import</span> <span class="nn">sys</span>
</span><span class='line'><span class="kn">import</span> <span class="nn">re</span>
</span></code></pre></td></tr></table></div></figure>


<p>You won’t need to type these two lines anymore in your interpreter. It doesn’t
change how Python executes files, just the interactive interpreter.</p>

<h2>Testing</h2>

<p>Four different kinds of tests are covered by this part: style checkers to
ensure your code’s consistency, static analysis tools to detect problems before
executing the code, unit tests to actually test your library, and code coverage
tests to ensure you do test <em>all</em> the code.</p>

<h3>Style Checking</h3>

<p>These are tools which help you maintaining a consistent coding style in your
whole codebase. Most of these tools are easy to use, the hardest part being to
choose which one fits your requirements.</p>

<p>Python has a PEP (<q>Python Enhancement Proposal</q>, sort of RFC), the
<a href="http://legacy.python.org/dev/peps/pep-0008/">PEP&nbsp;8</a>, dedicated to its coding conventions. If you want to follow
it, a command-line tool, rightly named <code>pep8</code>, is available.</p>

<pre><code>$ venv/bin/pip install pep8
$ venv/bin/pep8 your/lib/root/directory
...
foo/mod.py:84:80: E501 line too long (96 &gt; 79 characters)
foo/mod.py:85:6: E203 whitespace before ':'
foo/mod.py:86:80: E501 line too long (87 &gt; 79 characters)
foo/mod.py:87:4: E121 continuation line indentation is not a multiple of four
...
</code></pre>

<p>It’ll check each file and print a list of warnings. You can choose to hide some
of them, or use a white list to decide which ones you want. It’s a good tool if
you want to follow the PEP&nbsp;8 conventions.</p>

<p>Another highly customizable tool is <code>pylint</code>. It reads its configuration from
a file in your project, which can <a href="http://stackoverflow.com/a/22449845/735926">inherit</a> from global and user
configurations. It’ll warn you about bad naming, missing docstrings, functions
which take too many arguments, duplicated code, etc. It also gives you some
statistics about your code. It’s really powerful but can be a pain if you don’t
configure it. For example, it warns you about one-letter variables while you
might find them ok.</p>

<p>Enters <code>prospector</code>.</p>

<p>Prospector is built on top of <code>pep8</code> and <code>pylint</code> and comes with sane defaults
regarding the pickiness of both tools. You can <a href="http://blog.landscape.io/prospector-python-static-analysis-for-humans.html">tell it</a> to only
print important problems about your code:</p>

<pre><code>$ venv/bin/pip install prospector
$ venv/bin/prospector --strictness high
</code></pre>

<p>You’ll get a much shorter output, which will hopefully help you find potential
problems in your code.</p>

<h3>Static Analysis</h3>

<p>Here, we’re talking about analysing the code without executing it. Compiled
languages benefit from this at compilation time, but interpreted languages like
Python have no way to have it.</p>

<p>One of the most popular tools for static analysis on Python code is
<a href="https://launchpad.net/pyflakes">Pyflakes</a>. It doesn’t check your coding style like <code>pep8</code> or <code>pylint</code>, but
warns you about missing imports, dead code, unused variables, redefined
functions and more. You can work without style checkers, but static analysis is
really helpful to detect potential bugs before actually running the code.</p>

<p>Pyflakes can be integrated in editors like Vim with <a href="https://github.com/scrooloose/syntastic#readme">Syntastic</a>, but its
command-line usage is as easy as the previous tools:</p>

<pre><code>$ venv/bin/pip install pyflakes
$ venv/bin/pyflakes your/directory
</code></pre>

<p>Prospector, mentioned in the previous section, also includes <code>pyflakes</code>. You
might also want to try <a href="https://pypi.python.org/pypi/flake8">Flake8</a>, which combines <code>pyflakes</code> and <code>pep8</code>.</p>

<h3>Unit Tests</h3>

<p>When talking about <q>testing</q>, we usually think of <em>unit</em> testing, which is
testing small pieces of our code at a time, to make sure everything is working
correctly. The goal is to test only one feature at a time, to quickly be able
to find which parts of the code are not working. There are a lot of great
testing frameworks, and Python comes with a built-in one, <a href="https://docs.python.org/2/library/unittest.html">unittest</a>, which I
personnally use. I won’t cover these, and I’ll instead cover the case when you
need to test on multiple Python versions, which is often the case when you
plan to release a public library. You obviously don’t want to manually switch
to each Python version, install your dependencies then run your tests suit each
time.</p>

<p>This is a job for <code>tox</code>.</p>

<p><a href="https://testrun.org/tox/latest/">Tox</a> uses Virtualenv, which I talked about earlier, to create standalone
Python environments for different Python versions, and test your code in each
one of them.</p>

<pre><code>$ venv/bin/pip install tox
</code></pre>

<p>Like some previous tools, it needs a configuration file. Here is a basic
<code>tox.ini</code> to test on Python 2.6, 2.7, 3.3 and 3.4:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
</pre></td><td class='code'><pre><code class='ini'><span class='line'><span class="k">[tox]</span>
</span><span class='line'><span class="na">envlist</span> <span class="o">=</span> <span class="s">py26, py27, py33, py34</span>
</span><span class='line'><span class="na">downloadcache</span> <span class="o">=</span> <span class="s">{toxworkdir}/_download/</span>
</span><span class='line'>
</span><span class='line'><span class="k">[testenv]</span>
</span><span class='line'><span class="na">sitepackages</span> <span class="o">=</span> <span class="s">False</span>
</span><span class='line'><span class="na">deps</span> <span class="o">=</span><span class="s"></span>
</span><span class='line'><span class="s">  colorama</span>
</span><span class='line'><span class="na">commands</span> <span class="o">=</span><span class="s"></span>
</span><span class='line'><span class="s">  {envpython} {toxinidir}/tests/test.py</span>
</span></code></pre></td></tr></table></div></figure>


<p>It declares one dependency, <code>colorama</code>, and tells <code>tox</code> to run tests by
executing <code>tests/test.py</code>. That’s all. We can then run our tests:</p>

<pre><code>$ venv/bin/pip/tox
</code></pre>

<p>It’ll takes some time on the first run to fetch dependencies and create
environments, but all the following times will be faster.</p>

<p>Like <code>virtualenv</code>, <code>tox</code> uses a directory to store these environments. You can
safely delete it if you need more space, it’ll be re-created by <code>tox</code> the next
time:</p>

<pre><code>$ rm -rf .tox
</code></pre>

<h3>Code Coverage</h3>

<p>This last part about testing talks about code coverage tests. These are tests
about tests. The goal here is to ensure your tests cover all your code, and
you don’t leave some parts untested. Most tools tell you how many lines where
executed when running your tests suit, and give you an overall coverage
percentage.</p>

<p>One of them is <code>coverage</code>.</p>

<pre><code>$ venv/bin/pip install coverage
</code></pre>

<p>Give it your project’s root directory as well as a file to run your tests:</p>

<pre><code>$ venv/bin/coverage run --source=your/directory tests/test.py
</code></pre>

<p>It’ll run them, and give you a nice coverage report:</p>

<pre><code>$ coverage report -m
Name              Stmts   Miss  Cover   Missing
-----------------------------------------------
foobar/__init__       5      0   100%
foobar/base          46      2    96%   5-6
foobar/cli          159    159     0%   3-280
foobar/config        66     66     0%   3-150
foobar/session       64     14    78%   9-10, 106-124
foobar/barfooqux     22      0   100%
foobar/helloworld    25     20    20%   14-41
...
-----------------------------------------------
TOTAL               459    313    32%
</code></pre>

<p>It can also give you an HTML version:</p>

<pre><code>$ venv/bin/coverage html
</code></pre>

<p>Getting to 100% is the ultimate goal, but you’ll quickly find that’s the first
80% are easy and the remaining 20% are the hardest part, especially when you
have I/O, external dependencies like databases, and/or complicated
corner-cases.</p>

<p>Check <a href="http://nedbatchelder.com/code/coverage/">Coverage</a>’s doc for more info.</p>

<h2>Debugging</h2>

<p>There are a lot of ways to debug, including logs, but the simplest debugging
tool is the good old <code>print</code>. It becomes really impracticable when you have to
restart your server every time you add or remove one of them from your code.
What if you could fire a Python interpreter right from your code and inspect it
when it’s running? Well, you can do that with Python’s <code>code</code> module! This
trick is really handy, and I’ve been using it heavily instead of these <code>print</code>s
we write everywhere since I’ve discovered it.</p>

<p>The <code>code</code> module provides you with an <code>interact</code> function, which takes a
<code>dict</code> of variables to inject in the interpreter. You’ll be able to print them,
play with them, but these changes won’t be reflected in the program you’re
debugging.</p>

<p>Remember that Python lets you get all global variables as a <code>dict</code> with
<code>globals()</code> and all local ones with <code>locals()</code>. We thus start by creating a
mirror of the local environment:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="nb">vars</span> <span class="o">=</span> <span class="nb">globals</span><span class="p">()</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
</span><span class='line'><span class="nb">vars</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="nb">locals</span><span class="p">())</span>
</span></code></pre></td></tr></table></div></figure>


<p>These two lines get all global variables (including imported modules) in a
<code>dict</code> called <code>vars</code> and add local variables in it. This can then be passed
directly to the interpreter:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="kn">import</span> <span class="nn">code</span>
</span><span class='line'><span class="n">code</span><span class="o">.</span><span class="n">interact</span><span class="p">(</span><span class="n">local</span><span class="o">=</span><span class="nb">vars</span><span class="p">)</span>
</span></code></pre></td></tr></table></div></figure>


<p>This will start an interpreter with all these variables already available in
it. There’s nothing to install, this is a <a href="https://docs.python.org/2/library/code.html#code.interact">standard Python module</a>.</p>

<p>You can even inline the code and add it in your favorite snippets manager:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="n">g</span><span class="o">=</span><span class="nb">globals</span><span class="p">()</span><span class="o">.</span><span class="n">copy</span><span class="p">();</span><span class="n">g</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="nb">locals</span><span class="p">());</span><span class="kn">import</span> <span class="nn">code</span><span class="p">;</span><span class="n">code</span><span class="o">.</span><span class="n">interact</span><span class="p">(</span><span class="n">local</span><span class="o">=</span><span class="n">g</span><span class="p">)</span>
</span></code></pre></td></tr></table></div></figure>


<p>Don’t forget to remove it when you’re done!</p>

<h2>TL;DR</h2>

<ul>
<li>Use <code>virtualenv</code> to isolate your Python environment, <code>pip freeze</code> and a
<code>requirements.txt</code> file to keep track of your dependencies.</li>
<li>Write a <code>pythonrc.py</code> file to add autocomplete support to your interpreter</li>
<li>Use <code>pep8</code>, <code>pylint</code> and <code>pyflakes</code> to keep a high code quality</li>
<li>Use <code>tox</code> to test on multiple Python versions</li>
<li>Fire a local interpreter instead of <code>print</code>ing variables</li>
</ul>


<p>That was all. Please comment on this post if you think of any tool you use to
speed-up your Python development!</p>


feed_id: http://blog.bfontaine.net/2014/09/04/how-to-speed-up-vim-command-t-plugin
title: How to speed-up Vim’s Command-T plugin
uri: How-to-speed-up-Vim-s-Command-T-plugin
url: http://blog.bfontaine.net/2014/09/04/how-to-speed-up-vim-command-t-plugin/
content:
<p><a href="https://github.com/wincent/command-t#readme">Command-T</a> is a wonderful Vim plugin which allows you to open files
<q>with a minimal number of keystrokes</q>. It’s really handy in a large
codebase where you only have to type <code>&lt;leader&gt;t</code>, then a couple letters and
press enter to open your file. It’s based on a fuzzy matching, which let you
skip letters without worrying.</p>

<p>I recently installed the plugin on another machine and noticed it was really
low: I had to wait a couple seconds to get the files list everytime. My
computer has 8GB RAM so the problem wasn’t there.</p>

<!-- more -->


<p>The solution is pretty simple: the plugin relies on a C extension which I
forgot to compile after the installation.</p>

<p>From the docs:</p>

<pre><code>cd ~/.vim/bundle/command-t/ruby/command-t
ruby extconf.rb
make
</code></pre>

<p>It’ll make the plugin incredibly faster. This step is easy to miss if you read
the docs too quickly. I wrote this blog post to remember this, I hope it might
help a couple others since I didn’t find anything on Google about this issue.</p>


title: How to remember the difference between conj and cons in Clojure
uri: How-to-remember-the-difference-between-conj-and-cons-in-Clojure
feed_id:
http://blog.bfontaine.net/2014/05/25/how-to-remember-the-difference-between-conj-and-cons-in-clojure

url:
http://blog.bfontaine.net/2014/05/25/how-to-remember-the-difference-between-conj-and-cons-in-clojure/

content:
<p>When I started writing Clojure, I couldn’t memorize the difference between
<code>conj</code> and <code>cons</code> and always used one instead of the another. Their name are
similar, but <code>cons</code> is used to add an element at the beginning of a vector,
while <code>conj</code> is used to add an element at the end of it. How can one memorize
this? I found a mnemonic trick over the time that helps me remember this. Here
is it:</p>

<!-- more -->


<p>The trick is to look at the last letter of each function, <code>s</code> and <code>j</code>. As shown
in the image below, the <code>s</code> of <code>cons</code> shows the right, while the <code>j</code> of
<code>conj</code> shows the left.</p>

<p><img class="center" src="media/cc8f9d76.sj.png" data-original-source="http://blog.bfontaine.net/images/sj.png"></p>

<p>This means that <code>cons</code> pushes elements from the left to the right, that is,
<strong>at the beginning</strong> of a vector. <code>conj</code>, on the other hand, pushes elements
from the right to the left, which is <strong>at the end</strong> of a vector. That’s it.
Once you see this in your head, you’ll never forget the difference between
<code>cons</code> and <code>conj</code> on a vector.</p>


feed_id: http://blog.bfontaine.net/2014/03/03/fixing-gnuplot-pys-unknown-aqua-terminal-warning-on-osx
title: Fixing gnuplot-py’s “unknown aqua terminal” warning on OSX
uri: Fixing-gnuplot-py-s-unknown-aqua-terminal-warning-on-OSX
url: http://blog.bfontaine.net/2014/03/03/fixing-gnuplot-pys-unknown-aqua-terminal-warning-on-osx/
content:
<p>When plotting with <a href="https://pypi.python.org/pypi/gnuplot-py"><code>gnuplot-py</code></a> on OSX, I got an annoying warning
saying that terminal <code>aqua</code> is <q>unknown or ambiguous</q>, <em>even</em> when I use a
different terminal (e.g. <code>postscript</code>). This terminal doesn’t exist on my
Gnuplot installation (4.6.3). In fact, <code>gnuplot-py</code> <a href="https://github.com/jrk/gnuplot-py/blob/96b7c7ec81daa560325f4568393e300de020c353/gp.py#L28-39">uses</a> slightly
different files depending on your platform. OSX’s one is exactly the same as
other UNIX-flavored OSes but its default terminal is <code>aqua</code>. There are two ways
to fix the warning, a hacky one I used before this blog post, and a clean one I
discovered while writting this post. Hope this help!</p>

<!-- more -->


<h2>The hacky way</h2>

<p>Since OSX’s Gnuplot file is the same as Linux’s one apart from its default
term, we just need to tell <code>gnuplot-py</code> we’re on Linux:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
</pre></td><td class='code'><pre><code class='py'><span class='line'><span class="kn">import</span> <span class="nn">sys</span>
</span><span class='line'><span class="n">_platform</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span> <span class="c"># save the normal sys.platform value</span>
</span><span class='line'>
</span><span class='line'><span class="k">if</span> <span class="n">_platform</span> <span class="o">==</span> <span class="s">&#39;darwin&#39;</span><span class="p">:</span>
</span><span class='line'>    <span class="n">sys</span><span class="o">.</span><span class="n">platform</span> <span class="o">=</span> <span class="s">&#39;linux&#39;</span> <span class="c"># replace it with &#39;linux&#39;</span>
</span><span class='line'>
</span><span class='line'><span class="kn">import</span> <span class="nn">Gnuplot</span><span class="o">,</span> <span class="nn">Gnuplot.funcutils</span> <span class="c"># import Gnuplot</span>
</span><span class='line'><span class="n">sys</span><span class="o">.</span><span class="n">platform</span> <span class="o">=</span> <span class="n">_platform</span> <span class="c"># restore it</span>
</span></code></pre></td></tr></table></div></figure>


<p>It temporarily changes <code>sys.platform</code> value, which <code>gnuplot-py</code> <a href="https://github.com/jrk/gnuplot-py/blob/96b7c7ec81daa560325f4568393e300de020c353/gp.py#L28-39">is relying on</a>,
to <code>"linux"</code>, import Gnuplot and then restore it back to its previous value
(<code>"darwin"</code>).</p>

<h2>The clean way</h2>

<p>In fact there’s a much cleaner way to fix the warning. The default terminal is
stored in <a href="https://github.com/jrk/gnuplot-py/blob/96b7c7ec81daa560325f4568393e300de020c353/gp_unix.py#L86-91"><code>GnuplotOpts.default_term</code></a>, so we just need to change
it to another value to fix the warning:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
</pre></td><td class='code'><pre><code class='py'><span class='line'><span class="kn">import</span> <span class="nn">Gnuplot</span><span class="o">,</span> <span class="nn">Gnuplot.funcutils</span>
</span><span class='line'><span class="n">Gnuplot</span><span class="o">.</span><span class="n">GnuplotOpts</span><span class="o">.</span><span class="n">default_term</span> <span class="o">=</span> <span class="s">&#39;x11&#39;</span>
</span></code></pre></td></tr></table></div></figure>


<p>It’s cleaner than the hacky way because:</p>

<ol>
<li>It’s clear: the second line is pretty explicit: we change <code>default_term</code> to
<code>'x11'</code>. We don’t even need to add a comment.</li>
<li>It’s more maintainable: if OSX’s <code>gnuplot-py</code> interface change in the
future, this piece of code won’t break compatibility while the hacky way
will prevent our code to use the new interface.</li>
<li>It’s shorter: one line versus five ones. It’s really easier to introduce
bugs when using five lines instead of just one.</li>
</ol>


feed_id: http://blog.bfontaine.net/2014/02/15/using-coveralls-with-clojure
title: Using Coveralls with Clojure
uri: Using-Coveralls-with-Clojure
url: http://blog.bfontaine.net/2014/02/15/using-coveralls-with-clojure/
content:
<p><a href="https://coveralls.io/"><strong>Coveralls</strong></a> is a service that keep track of your tests coverage
for you. It can notifies you when your coverage decreases under a custom
threshold, and their bot comments on pull requests to report their tests
coverage. Like Travis-CI, it allows you to add a badge to your readme with
an up-to-date tests coverage percentage.</p>

<p>If you already test your GitHub projects with a CI server like Travis, it’s
very easy to add Coveralls to your workflow. Unfortunately, they have a library
for Ruby, a couple user-provided libraries for other languages such as PHP,
Java and Python, but nothing for Clojure. Fortunately, they provide an API for
unsupported languages like Clojure. Here is how to use it.</p>

<!-- more -->


<h2>Cloverage</h2>

<p><a href="https://github.com/lshift/cloverage"><strong>Cloverage</strong></a> is a Clojure library and a Leiningen plugin to get
form-level tests coverage of your projects. It’s dead easy to use and generates
HTML reports for you as well as some other formats.</p>

<p>Add their plugin to your <code>project.clj</code>:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
</pre></td><td class='code'><pre><code class='clj'><span class='line'><span class="p">(</span><span class="kd">defproject </span><span class="nv">your-project</span> <span class="s">&quot;0.1.0&quot;</span>
</span><span class='line'>  <span class="c1">; ...</span>
</span><span class='line'>  <span class="ss">:plugins</span> <span class="p">[[</span><span class="nv">lein-cloverage</span> <span class="s">&quot;1.0.2&quot;</span><span class="p">]])</span>
</span></code></pre></td></tr></table></div></figure>


<p>Use <code>lein cloverage</code> to run your tests and generate various reports. You can
specify which Cloverage version you want to use with <code>CLOVERAGE_VERSION</code>. The
latest stable one is <code>1.0.3</code>, but we’ll need to use <code>1.0.4-SNAPSHOT</code> for this
article:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='sh'><span class='line'><span class="nv">CLOVERAGE_VERSION</span><span class="o">=</span>1.0.4-SNAPSHOT lein cloverage
</span></code></pre></td></tr></table></div></figure>


<h2>Coveralls</h2>

<p>If you don’t already use it, <a href="https://coveralls.io/authorize/github">signup</a> on Coveralls using your GitHub account.
Then, click on “Add Repo”, find your repo in the list and activate it. That’s
all you have to do on their website for now.</p>

<h2>Travis CI</h2>

<p>You now need to custom your builds on Travis to use Coveralls. We’ll do it in a
Bash script, so add this to your <code>.travis.yml</code>:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
</pre></td><td class='code'><pre><code class='yaml'><span class='line'><span class="l-Scalar-Plain">after_script</span><span class="p-Indicator">:</span>
</span><span class='line'>  <span class="p-Indicator">-</span> <span class="l-Scalar-Plain">bash -ex test/coveralls.sh</span>
</span></code></pre></td></tr></table></div></figure>


<p>This tells Travis to run <code>test/coveralls.sh</code> after each build. We use Bash
options <code>-e</code> and <code>-x</code> to respectively stop the script at the first failure and
print each command.</p>

<p>Now edit <code>test/coveralls.sh</code> and add the following content:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
</pre></td><td class='code'><pre><code class='bash'><span class='line'><span class="nv">COVERALLS_URL</span><span class="o">=</span><span class="s1">&#39;https://coveralls.io/api/v1/jobs&#39;</span>
</span><span class='line'><span class="nv">CLOVERAGE_VERSION</span><span class="o">=</span><span class="s1">&#39;1.0.4-SNAPSHOT&#39;</span> lein2 cloverage -o cov --coveralls
</span><span class='line'>curl -F <span class="s1">&#39;json_file=@cov/coveralls.json&#39;</span> <span class="s2">&quot;$COVERALLS_URL&quot;</span>
</span></code></pre></td></tr></table></div></figure>


<p>It’ll run Cloverage and output reports in <code>cov</code> (use another name if you
prefer), using a report for Coveralls. This generates a <code>coveralls.json</code> file
which can then be sent to their API. That’s what we do on the next line, using
cURL.</p>

<p>That’s all! You can now push on GitHub, Travis will run your tests and send
their tests coverage to Coveralls.</p>

<h2>Caveats</h2>

<p>Coveralls doesn’t support partial-line tests coverage, so we’re cheating a
little bit here using hits count. A line that it never covered has 0 hits, one
that is partially covered has one hit, and a fully covered one has two hits.
There’s <a href="https://github.com/lemurheavy/coveralls-public/issues/216">an open issue</a> regarding this.</p>


feed_id: http://blog.bfontaine.net/2014/02/09/function-level-black-box-testing
title: Function-level Black-box Testing
uri: Function-level-Black-box-Testing
url: http://blog.bfontaine.net/2014/02/09/function-level-black-box-testing/
content:
<p>Black-box testing <q>is a method of software testing that examines the
functionality of an application (e.g. what the software does) without peering
into its internal structures or workings</q> (<a href="https://en.wikipedia.org/wiki/Black-box_testing">Wikipedia</a>). While it’s
usually done at a system level, I think the most obvious place it should be
used is at the function level. It’s even more efficient if you write tests for
someone else’s code.<br/>
Here is how I write unit tests for functions.</p>

<!-- more -->


<h2>Good Names</h2>

<p>If you can’t tell what the function roughly does only by looking at its name,
it’s usually because it’s badly named or it’s doing too many things, maybe you
should break it in smaller functions. Look at its arguments names. Do they make
sense to you? Do you <em>understand</em> what’s the function doing only by looking as
its name and its arguments?</p>

<h2>Good Documentation</h2>

<p>You sometimes need to read a function’s documentation if its name is unclear or
if it’s a complicated code area. This is your latest chance to understand the
function, because in function-level black-box testing you can’t read the code,
it would not be black-box anymore. If you understand the function, you can now
write tests. If not, you should improve its name, its arguments names, and its
documentation, in that order because it’s the order one reads it.</p>

<h2>Good Tests</h2>

<p>Good unit tests are short, incremental, and test only a specific case. Start
with the simplest. Does the function takes a string? Give it an empty one. Does
it take a number? Give it <code>0</code>. Use <code>null</code> for objects. Then move to more edge
cases: negative numbers, un-trimmed strings, uninitialized objects. Add more
simple edge cases, but only once at a time. Then, combine those edge cases.
Write more complex cases. Never combine two cases you never individually tested
before. After all these edge cases, test the legitimate ones. Again start with
the simplest, then add more complexity and combine cases.</p>


feed_id: http://blog.bfontaine.net/2013/10/22/local-vimrcs
title: Local .vimrc's
uri: Local-vimrc-s
url: http://blog.bfontaine.net/2013/10/22/local-vimrcs/
content:
<p>I <a href="http://damien.lespiau.name/blog/2009/03/18/per-project-vimrc/">discovered</a> a great feature in Vim today. While we generally
use the same Vim settings everywhere, we sometimes (have to) contribute to
projects with a coding style different of ours, and because nobody wants to
edit their <code>.vimrc</code> everytime to switch between configurations, Vim allows you
to have per project <code>.vimrc</code>&rsquo;s.</p>

<!--more-->


<p>This feature can be enabled with one line in your <code>.vimrc</code>:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='vim'><span class='line'><span class="k">set</span> <span class="nb">exrc</span>
</span></code></pre></td></tr></table></div></figure>


<p>When you start Vim, it begins by looking at some predefined locations for a
configuration file (the first one is <code>$HOME/.vimrc</code>).  The <code>exrc</code> option,
disabled by default, tell to Vim to not only load your personal <code>.vimrc</code>, but
also to look in the current directory to check if there’s another <code>.vimrc</code> file,
which will be loaded <em>after</em> your personal one. You can then use it to override
your settings and/or add specific ones for this directory.</p>

<p>Additionally, if you share the local <code>.vimrc</code> with others (e.g. in a Git
repository), you may want to set the <code>secure</code> option at the end of your personal
<code>.vimrc</code>:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='vim'><span class='line'><span class="k">set</span> <span class="nb">secure</span>
</span></code></pre></td></tr></table></div></figure>


<p>This disallow shell and write commands in the <code>.vimrc</code> of the local directory.
On <span class="sc">unix</span>-like OSes, this option is set by default if
you’re not the owner of the file.</p>

<p>For more Vim gems, check my <code>.vimrc</code> on <a href="https://github.com/bfontaine/Dotfiles/blob/master/.vimrc">GitHub</a>!</p>


feed_id: http://blog.bfontaine.net/2013/08/26/use-a-custom-tld-for-local-development
title: Use a Custom Tld for Local Development
uri: Use-a-Custom-Tld-for-Local-Development
url: http://blog.bfontaine.net/2013/08/26/use-a-custom-tld-for-local-development/
content:
<p>In this post, we’ll create a custom <a href="https://en.wikipedia.org/wiki/Top-level_domain">TLD</a> for local development, and configure
Apache to work with that. It’ll allow you to work on your local version of
<code>mysuperwebsite.com</code> with the local domain <code>mysuperwebsite.dev</code>, with the exact
same URLs, except that little <code>.com</code> which is replaced by <code>.dev</code>.</p>

<!--more-->


<p>This post is focused on OS X, but the tools used are available on Ubuntu and
others. Additionally, you can choose whatever unexisting domain you want, just
replace <code>.dev</code> with the one you chose in the commands described in the article.</p>

<h2>DNSMasq</h2>

<p>The major problem we have when we develop locally with custom domains is that
we have to add an entry to <code>/etc/hosts</code> for <em>each</em> website. Wildcards are not
supported, so you can’t write the following line in it:</p>

<pre><code>127.0.0.1 *.dev
</code></pre>

<p>There are different workarounds, but here we’ll use DNSMasq as
a local DNS resolver. If you don’t have <a href="http://brew.sh/">homebrew</a>, install it
first, then install DNSMasq:</p>

<pre><code>brew install dnsmasq
# enable the daemon on startup
sudo cp $(brew --prefix dnsmasq)/homebrew.mxcl.dnsmasq.plist /Library/LaunchDaemons/
</code></pre>

<p>DNSMasq will run locally and redirect any query for a <code>*.dev</code> domain to the local
host, <code>127.0.0.1</code>. Open <code>/usr/local/etc/dnsmasq.conf</code> (or create it if it doesn’t
exist) and add the following lines:</p>

<pre><code>address=/dev/127.0.0.1
listen-adress=127.0.0.1
</code></pre>

<p>Then start DNSMasq:</p>

<pre><code>sudo launchctl load homebrew.mxcl.dnsmasq
</code></pre>

<p>You’ll have to tell OS X to send its
DNS queries to the local server first, then try the other ones, to intercept
the queries for the <code>*.dev</code> domains. Go to <i>System Preferences → Network →
Advanced → DNS</i>, and add <code>127.0.0.1</code> at the top of the list of DNS servers.</p>

<p>You can then check that it works using <code>dig</code>:</p>

<pre><code>$ dig foobar.dev
…
;; QUESTION SECTION:
;foobar.dev.            IN  A

;; ANSWER SECTION:
foobar.dev.     0   IN  A   127.0.0.1

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
…
</code></pre>

<p>If you have some
server listening on port 80, you can try <code>foobar.dev</code> in your browser, it’ll
display whatever you’re serving on <code>127.0.0.1:80</code>. If you have any troubles,
empty your cache (use <code>dscacheutils -flushcache</code> on OS X), or restart your
computer.</p>

<p>That’s all! You can stop there, but if you’re using Apache you may be
interested by the next section of this article.</p>

<h2><code>mod_vhost_alias</code></h2>

<p>This Apache module allows you manage virtual hosts dynamically, so you won’t
have to create a new one for every (local) website. This is useful if you have
a large number of virtual hosts with similar configurations. In this section,
we’ll see how to associate a <code>.dev</code> domain with a directory on your computer.
We’ll assume you already have Apache installed and working.</p>

<p>With <code>mod_vhost_alias</code>, Apache extracts the hostname from the client query (with
the <code>Host</code> HTTP parameter) and use it to get the directory path. The <a href="https://httpd.apache.org/docs/current/mod/mod_vhost_alias.html">official doc</a>
has a lot of examples, I personally prefer to be able to use whatever directory
name I want, so I’m using a <code>vhosts</code> directory which contains symbolic links to
the right directories. Like with DNSMasq before, you need to add only two lines
of configuration here. Open <code>/etc/apache2/extra/httpd-vhosts.conf</code> (you’ll need
to use <code>sudo</code>), and add the following lines at the end of it:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
</pre></td><td class='code'><pre><code class='apache'><span class='line'><span class="nb">UseCanonicalName</span> <span class="k">Off</span>
</span><span class='line'><span class="nb">VirtualDocumentRoot</span> <span class="s2">&quot;/Users/baptiste/some/dirs/vhosts/%-2.0.%-1.0&quot;</span>
</span></code></pre></td></tr></table></div></figure>


<p>You may want to customize the path. This one takes the last two parts of the
domain (i.e. the domain and the tld), and use the directory
<code>~/some/dirs/vhosts/domain.tld/</code> for it. Note that it’ll use the same directory
for <code>bar.dev</code>, <code>foo.bar.dev</code>, <code>qux.foo.bar.dev</code>, etc.</p>

<p>Then, use <code>ln -s</code> to make symbolic links from <code>vhosts/</code> to the right directories,
e.g.:</p>

<pre><code>~/some/dirs/vhosts/foo.dev -&gt; ~/sites/foo.dev
~/some/dirs/vhosts/bar.dev -&gt; ~/perso/blog
</code></pre>

<p>If all the websites are in the same directory, you can skip the
“symbolic links” part. Restart Apache with <code>sudo apachectl restart</code> (on Ubuntu,
use <code>sudo service apache2 restart</code>), and you’re done. In the future, you won’t
have to restart Apache for each new site, you need to restart it only when you
modify its configuration.</p>

<p>If you get some issues with rewrite rules, add</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='apache'><span class='line'><span class="nb">RewriteBase</span> /
</span></code></pre></td></tr></table></div></figure>


<p>in the <code>.htaccess</code> of the websites that use them, and it’ll work.</p>


feed_id: http://blog.bfontaine.net/2013/08/11/get-rfcs-in-your-terminal
title: Get RFCs in Your Terminal
uri: Get-RFCs-in-Your-Terminal
url: http://blog.bfontaine.net/2013/08/11/get-rfcs-in-your-terminal/
content:
<p>When working with Internet protocols, we have to read <a href="http://www.ietf.org/rfc.html">RFCs</a> a lot. They
can be found on the Web, but it’s better to have them directly in the terminal.
Ubuntu provide <a href="https://launchpad.net/ubuntu/+source/doc-rfc">some packages</a> to have them offline, but if you aren’t a
sudoer, you can’t install them with <code>apt-get</code>. So I needed a little script to
fetch RFCs from IETF’s website and read them locally.</p>

<!--more-->


<h2>Here comes rfc</h2>

<p><code>rfc</code> was initially a small Bash script (~5 lines) that <code>cURL</code>-ed RFCs and
displayed them with <code>less</code>. I used it for the Networking class at Paris Diderot.
A few weeks ago, I enhanced it with a local cache (it now download an RFC the
first time only), and an offline search feature. Thanks to <a href="https://github.com/ecksun">ecksun</a>, it
can also be used to read drafts. The script works pretty much everywhere, and is
really simple to use:</p>

<pre><code>$ rfc &lt;number&gt;
</code></pre>

<p>For example, get the RFC 6749 (OAuth 2.0) with:</p>

<pre><code>$ rfc 6749
</code></pre>

<p>That’s all! Since it’s just plain text, you can pipe it or redirect its output
to anything:</p>

<pre><code>$ rfc 42 | lolcat        # rainbow RFC
$ rfc 4534 &gt; rfc4534.txt # local copy
</code></pre>

<h3>Install</h3>

<p>Since that’s a standalone Bash script, you can put it where you want, but the
directory must be in your <code>PATH</code>. Here is a basic install:</p>

<pre><code>mkdir -p ~/bin
curl https://raw.github.com/bfontaine/rfc/master/rfc &gt; ~/bin/rfc
chmod u+x ~/bin/rfc
</code></pre>

<p>If you don’t have <code>~/bin</code> in your <code>PATH</code>, add this line in your <code>~/.bashrc</code>:</p>

<pre><code>export PATH="$HOME/bin:$PATH"
</code></pre>

<p>The only requirements are a pager (<code>less</code> is the default, but it’ll use <code>$PAGER</code>
if it’s set) and <code>curl</code> (it’ll use <code>$CURL</code> if it’s set, and fallback on <code>wget</code>
if <code>curl</code> can’t be found).</p>

<p>For more info, check the project on <a href="https://github.com/bfontaine/rfc">GitHub</a>.</p>


feed_id: http://blog.bfontaine.net/2013/07/23/open-spotify-links-with-the-desktop-client
title: Open Spotify links with the Desktop client
uri: Open-Spotify-links-with-the-Desktop-client
url: http://blog.bfontaine.net/2013/07/23/open-spotify-links-with-the-desktop-client/
content:
<p>Spotify’s <a href="https://play.spotify.com">Web app</a> is great, but you may prefer to
use the desktop client. The problem is that the links default to the Web app.
Here is a quick tip to make what you want.</p>

<!--more-->


<p>First, install <a href="https://chrome.google.com/webstore/detail/switcheroo-redirector/cnmciclhnghalnpfhhleggldniplelbg">Switcheroo</a>. It’s a Chrome extension (Firefox users,
go <a href="https://addons.mozilla.org/en-us/firefox/addon/redirector/">here</a>) that allows you
to setup custom redirect rules for any http request, using a string
replacement. Then, add a rule to replace <code>http://open.spotify.com/</code> with
<code>spotify://</code>.</p>

<p>That’s it! Now, all <code>open.spotify.com/something</code> links will open in the desktop
client instead of the Web one.</p>

<p><i class="small">Note: I found the original tip <a href="http://community.spotify.com/t5/Help-Web-Player/PLEASE-allow-users-to-default-to-desktop-client/m-p/460950/highlight/true#M2231">here</a>, but it was not working because the
author suggested to replace with <code>spotify</code> instead of <code>spotify://</code>.</i></p>


feed_id: http://blog.bfontaine.net/2013/06/04/get-the-urban-dictionary-in-your-terminal
title: Get the Urban Dictionary in your terminal
uri: Get-the-Urban-Dictionary-in-your-terminal
url: http://blog.bfontaine.net/2013/06/04/get-the-urban-dictionary-in-your-terminal/
content:
<p>The <a href="http://www.urbandictionary.com/">Urban Dictionary</a> website is pretty useful when one wants to
know the meaning of any familiar or slang word. Users can write definitions for
any words, and upvote or downvote them. Unfortunately, the website doesn’t have
a public API, and I couldn’t find a command-line tool for it. So I wrote one.</p>

<!--more-->


<h2>Introducing ud</h2>

<p><strong>ud</strong> is a command-line tool which prints definitions from the Urban Dictionary
website. It’s <a href="https://github.com/bfontaine/ud">written</a> in Ruby, and allows you to quickly search for a
definition of any word.</p>

<pre><code>$ ud wth
* WTH (567/126):

   1) Abbreviation for "What the Hell"
   2) Shortened alternative to "With"

 Example:
   1) WTH was that?!
   2) I ate some crackers wth chicken
(…)
</code></pre>

<p>For common abbreviations like “wtf” or “omg”, there are dozens of definitions,
with minor differences. That’s why the script scraps only the first page of
results, and supports a few options: <code>-n</code> allows you to limit the number of
definitions (e.g. <code>-n 1</code> → only one). <code>-r</code> allows you to select only the
definitions which have an upvotes/downvotes ratio higher than what you want,
e.g. <code>-r 2</code> will select only the definitions which have twice more upvotes than
downvotes. For your convenience, it also support the shortcut <code>-u</code> which is an
alias for <code>-r 2</code>. Type <code>ud -h</code> for more info.</p>

<h3>Install</h3>

<p>That’s easy:</p>

<pre><code>gem install ud
</code></pre>

<p>Please note that if you’re using Windows, you’ll need to install the Win32
Console ANSI gem for the colored output. Install it with:</p>

<pre><code>gem install win32console
</code></pre>

<h3>Using the UD module</h3>

<p>The gem also provide a <code>UD</code> module, which you can use to query the website in
your own Ruby code:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="nb">require</span> <span class="s1">&#39;ud&#39;</span>
</span><span class='line'>
</span><span class='line'><span class="n">wtf</span> <span class="o">=</span> <span class="no">UD</span><span class="o">.</span><span class="n">query</span> <span class="s1">&#39;wtf&#39;</span><span class="p">,</span> <span class="ss">:count</span> <span class="o">=&gt;</span> <span class="mi">1</span>
</span><span class='line'>
</span><span class='line'><span class="nb">puts</span> <span class="n">wtf</span><span class="o">[</span><span class="mi">0</span><span class="o">].</span><span class="n">keys</span><span class="o">.</span><span class="n">join</span> <span class="s1">&#39;, &#39;</span>
</span><span class='line'><span class="c1">#=&gt; id, word, definition, example, upvotes, downvotes</span>
</span></code></pre></td></tr></table></div></figure>


feed_id: http://bfontaine.net/blog/2015/10/29/a-javascript-modules-manager-that-fits-in-a-tweet
title: A JavaScript Modules Manager That Fits in a Tweet
uri: A-JavaScript-Modules-Manager-That-Fits-in-a-Tweet
url: http://bfontaine.net/blog/blog/2015/10/29/a-javascript-modules-manager-that-fits-in-a-tweet/
content:
<p>ES6 does provide <a href="http://es6-features.org/#ValueExportImport">modules</a>; but unless you’re using <a href="https://babeljs.io/">Babel</a>
you’ll have to rely on third-party libraries such as <a href="http://requirejs.org/">RequireJS</a> until all
major browsers support them.</p>

<p>I use <a href="http://d3js.org/">D3</a> everyday to visualize data about ego networks and
have a small (400-500 SLOC) JavaScript codebase I need to keep organized.
In the context I work in I must keep things simple as I won’t always be there
to maintain the code I’m writing today.</p>

<p>How simple a modules implementation could possibly be? It should at least be
able to register modules and require a module inside another; much like
Python’s <code>import</code>. It should also handle issues like circular dependencies
(e.g. <code>foo</code> requires <code>bar</code> which requires <code>foo</code>) and undeclared modules.
Modules should be lazily loaded, i.e. only when they are required; and
requiring twice the same module shouldn’t execute it twice.</p>

<p>Well, here is one:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='js'><span class="nx">p</span><span class="o">=</span><span class="p">{</span><span class="nx">f</span><span class="o">:</span><span class="p">{},</span><span class="nx">m</span><span class="o">:</span><span class="p">{},</span><span class="nx">r</span><span class="o">:</span><span class="kd">function</span><span class="p">(</span><span class="nx">a</span><span class="p">,</span><span class="nx">b</span><span class="p">){</span><span class="nx">p</span><span class="p">.</span><span class="nx">f</span><span class="p">[</span><span class="nx">a</span><span class="p">]</span><span class="o">=</span><span class="nx">b</span><span class="p">},</span><span class="nx">g</span><span class="o">:</span><span class="kd">function</span><span class="p">(</span><span class="nx">a</span><span class="p">){</span><span class="k">if</span><span class="p">(</span><span class="o">!</span><span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">a</span><span class="p">]){</span><span class="k">if</span><span class="p">(</span><span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">a</span><span class="p">]</span><span class="o">&lt;</span><span class="mi">1</span><span class="o">|!</span><span class="nx">p</span><span class="p">.</span><span class="nx">f</span><span class="p">[</span><span class="nx">a</span><span class="p">])</span><span class="k">throw</span><span class="s2">&quot;p:&quot;</span><span class="o">+</span><span class="nx">a</span><span class="p">;</span><span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">a</span><span class="p">]</span><span class="o">=</span><span class="mi">0</span><span class="p">;</span><span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">a</span><span class="p">]</span><span class="o">=</span><span class="nx">p</span><span class="p">.</span><span class="nx">f</span><span class="p">[</span><span class="nx">a</span><span class="p">](</span><span class="nx">p</span><span class="p">)}</span><span class="k">return</span> <span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">a</span><span class="p">]}};</span>
</code></pre></div></figure></notextile></div>


<p>It’s 136-bytes long. 139 if you count the variable definition. At this level
you can’t expect long function names but here is an usage example:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='js'><span class="c1">// register a &quot;main&quot; module. A module consists of a name and a</span>
<span class="c1">// function that takes an object used to require other modules.</span>
<span class="nx">p</span><span class="p">.</span><span class="nx">r</span><span class="p">(</span><span class="s2">&quot;main&quot;</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">r</span><span class="p">)</span> <span class="p">{</span>
    <span class="c1">// get the &quot;num&quot; module and store it in a `num` variable</span>
    <span class="kd">var</span> <span class="nx">num</span> <span class="o">=</span> <span class="nx">r</span><span class="p">.</span><span class="nx">g</span><span class="p">(</span><span class="s2">&quot;num&quot;</span><span class="p">);</span>

    <span class="c1">// use it to print something</span>
    <span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">num</span><span class="p">.</span><span class="nx">add</span><span class="p">(</span><span class="mi">20</span><span class="p">,</span> <span class="mi">22</span><span class="p">));</span>
<span class="p">});</span>

<span class="c1">// register a &quot;num&quot; module</span>
<span class="nx">p</span><span class="p">.</span><span class="nx">r</span><span class="p">(</span><span class="s2">&quot;num&quot;</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">r</span><span class="p">)</span> <span class="p">{</span>
    <span class="c1">// a module can export bindings by returning an object</span>
    <span class="k">return</span> <span class="p">{</span>
        <span class="nx">add</span><span class="o">:</span> <span class="kd">function</span><span class="p">(</span><span class="nx">a</span><span class="p">,</span> <span class="nx">b</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nx">a</span><span class="o">+</span><span class="nx">b</span><span class="p">;</span> <span class="p">},</span>
    <span class="p">};</span>
<span class="p">});</span>

<span class="c1">// call the &quot;main&quot; module</span>
<span class="nx">p</span><span class="p">.</span><span class="nx">g</span><span class="p">(</span><span class="s2">&quot;main&quot;</span><span class="p">);</span>
</code></pre></div></figure></notextile></div>


<p>This code will print <code>42</code> in the console. It only uses two modules but the
implementation works with an arbitrary number of modules. A module can depend
on any number of other modules that can be declared in an arbitrary order.</p>

<p>Consider this example:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='js'><span class="nx">p</span><span class="p">.</span><span class="nx">r</span><span class="p">(</span><span class="s2">&quot;m1&quot;</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">r</span><span class="p">)</span> <span class="p">{</span> <span class="nx">r</span><span class="p">.</span><span class="nx">g</span><span class="p">(</span><span class="s2">&quot;m2&quot;</span><span class="p">);</span> <span class="p">});</span>
<span class="nx">p</span><span class="p">.</span><span class="nx">r</span><span class="p">(</span><span class="s2">&quot;m2&quot;</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">r</span><span class="p">)</span> <span class="p">{</span> <span class="nx">r</span><span class="p">.</span><span class="nx">g</span><span class="p">(</span><span class="s2">&quot;m3&quot;</span><span class="p">);</span> <span class="p">});</span>
<span class="nx">p</span><span class="p">.</span><span class="nx">r</span><span class="p">(</span><span class="s2">&quot;m3&quot;</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">r</span><span class="p">)</span> <span class="p">{</span> <span class="nx">r</span><span class="p">.</span><span class="nx">g</span><span class="p">(</span><span class="s2">&quot;m1&quot;</span><span class="p">);</span> <span class="p">});</span>

<span class="nx">p</span><span class="p">.</span><span class="nx">g</span><span class="p">(</span><span class="s2">&quot;m1&quot;</span><span class="p">);</span>
</code></pre></div></figure></notextile></div>


<p><code>m1</code> depends on <code>m2</code> which depends on <code>m3</code> which itself depends on <code>m1</code>. The
implementation won’t die in an endless loop leading to a stack overflow but
will fail as soon as it detects the loop:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='js'><span class="nx">p</span><span class="o">:</span><span class="nx">m1</span>
</code></pre></div></figure></notextile></div>


<p>Admittedly this error message doesn’t give us too information but we have to be
thrifty in order to fit under 140 characters. The prefix <code>p:</code> tells you the
error comes from <code>p</code>, and the part after is the faulty module. It can either be
a wrong name (the module doesn’t exist) or a circular dependency.</p>

<h2>Walk-through</h2>

<p>Note: don’t use this at home. This is just an experiment; I eventually used
<a href="browserify.org">Browserify</a> for my project.</p>

<p>We need an object to map modules to their functions; we’ll populate it on calls
to <code>register</code>. We need another object to store the result of their function
call; i.e. what they export. I added a third object to “lock” a module while
it’s executed in order to detect circular dependencies.</p>

<p>We’ll have something like that:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='js'><span class="kd">var</span> <span class="nx">p</span> <span class="o">=</span> <span class="p">{</span>
    <span class="nx">_fn</span><span class="o">:</span> <span class="p">{},</span>   <span class="c1">// the functions</span>
    <span class="nx">_m</span><span class="o">:</span> <span class="p">{},</span>    <span class="c1">// the modules’ exported values</span>
    <span class="nx">_lock</span><span class="o">:</span> <span class="p">{},</span> <span class="c1">// the locks</span>

    <span class="nx">register</span><span class="o">:</span> <span class="kd">function</span><span class="p">(</span><span class="nx">name</span><span class="p">,</span> <span class="nx">callback</span><span class="p">)</span> <span class="p">{</span>
        <span class="c1">// add the function in the object</span>
        <span class="nx">p</span><span class="p">.</span><span class="nx">_fn</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span> <span class="o">=</span> <span class="nx">callback</span><span class="p">;</span>
    <span class="p">},</span>

    <span class="nx">get</span><span class="o">:</span> <span class="kd">function</span><span class="p">(</span><span class="nx">name</span><span class="p">)</span> <span class="p">{</span>
        <span class="c1">// if we have a value for this module let’s return it.</span>
        <span class="c1">// Note that we should use `.hasOwnProperty` here</span>
        <span class="c1">// because this’ll fail if the module returns a falsy</span>
        <span class="c1">// value. This is not really important for this problem.</span>
        <span class="k">if</span> <span class="p">(</span><span class="nx">p</span><span class="p">.</span><span class="nx">_m</span><span class="p">[</span><span class="nx">name</span><span class="p">])</span> <span class="p">{</span>
            <span class="k">return</span> <span class="nx">p</span><span class="p">.</span><span class="nx">_m</span><span class="p">[</span><span class="nx">name</span><span class="p">];</span>
        <span class="p">}</span>

        <span class="c1">// if it’s locked that’s because we’re already getting</span>
        <span class="c1">// it; so there’s a recursive requirement</span>
        <span class="k">if</span> <span class="p">(</span><span class="nx">p</span><span class="p">.</span><span class="nx">_lock</span><span class="p">[</span><span class="nx">name</span><span class="p">])</span> <span class="p">{</span>
            <span class="k">throw</span> <span class="s2">&quot;Recursive requirement: &#39;&quot;</span> <span class="o">+</span> <span class="nx">name</span> <span class="o">+</span> <span class="s2">&quot;&#39;&quot;</span><span class="p">;</span>
        <span class="p">}</span>

        <span class="c1">// if we don’t have any function for this we can’t</span>
        <span class="c1">// execute it and get its value. See also the</span>
        <span class="c1">// remark about `.hasOwnProperty` above.</span>
        <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">p</span><span class="p">.</span><span class="nx">_fn</span><span class="p">[</span><span class="nx">name</span><span class="p">])</span> <span class="p">{</span>
            <span class="k">throw</span> <span class="s2">&quot;Unknown module &#39;&quot;</span> <span class="o">+</span> <span class="nx">name</span> <span class="o">+</span> <span class="s2">&quot;&#39;&quot;</span><span class="p">;</span>
        <span class="p">}</span>

        <span class="c1">// we lock the module so we can detect circular</span>
        <span class="c1">// requirements.</span>
        <span class="nx">p</span><span class="p">.</span><span class="nx">_lock</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span> <span class="o">=</span> <span class="kc">true</span><span class="p">;</span>

        <span class="k">try</span> <span class="p">{</span>
            <span class="c1">// execute the module&#39;s function and pass</span>
            <span class="c1">// ourselves to it so it can require other</span>
            <span class="c1">// modules with p.get.</span>
            <span class="nx">p</span><span class="p">.</span><span class="nx">_m</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span> <span class="o">=</span> <span class="nx">p</span><span class="p">.</span><span class="nx">_fn</span><span class="p">[</span><span class="nx">name</span><span class="p">](</span><span class="nx">p</span><span class="p">);</span>
        <span class="p">}</span> <span class="k">finally</span> <span class="p">{</span>
            <span class="c1">// ensure we *always* remove the lock.</span>
            <span class="k">delete</span> <span class="nx">p</span><span class="p">.</span><span class="nx">_lock</span><span class="p">[</span><span class="nx">name</span><span class="p">];</span>
        <span class="p">}</span>

        <span class="c1">// return the result</span>
        <span class="k">return</span> <span class="nx">p</span><span class="p">.</span><span class="nx">_m</span><span class="p">[</span><span class="nx">name</span><span class="p">];</span>
    <span class="p">},</span>
<span class="p">};</span>
</code></pre></div></figure></notextile></div>


<p>This works and is pretty short; but that won’t fit in a Tweet ;)</p>

<p>Let’s compact the exceptions into one because those strings take a lot of
place:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='js'><span class="k">if</span> <span class="p">(</span><span class="nx">p</span><span class="p">.</span><span class="nx">_lock</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span> <span class="o">||</span> <span class="o">!</span><span class="nx">p</span><span class="p">.</span><span class="nx">_fn</span><span class="p">[</span><span class="nx">name</span><span class="p">])</span> <span class="p">{</span>
    <span class="k">throw</span> <span class="s2">&quot;Module error: &quot;</span> <span class="o">+</span> <span class="nx">name</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></figure></notextile></div>


<p>The error is less explicit but we&rsquo;ll accept that here.</p>

<p>We try to get as little code as possible then use <a href="https://yui.github.io/yuicompressor/">YUI&nbsp;Compressor</a> to
remove the spaces and rename the variables. This means we can still work with
(mostly) readable code and let YUI&nbsp;Compressor do the rest for us.</p>

<p>I measure the final code size with the following command:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='sh'>yuicompressor p.js <span class="p">|</span> wc -c
</code></pre></div></figure></notextile></div>


<p>Right now we have 240 bytes. We need a way to remove 100 bytes. Let’s rename
the attributes. <code>_fn</code> becomes <code>f</code>; <code>_m</code> becomes <code>m</code>, <code>_lock</code> becomes <code>l</code> and
the public methods are reduced to their first letter. We can also remove the
<code>var</code> since <code>p</code> will be global anyway. Let’s also reduce the error message
prefix to <code>"p:"</code>.</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='js'><span class="nx">p</span> <span class="o">=</span> <span class="p">{</span>
    <span class="nx">f</span><span class="o">:</span> <span class="p">{},</span> <span class="nx">m</span><span class="o">:</span> <span class="p">{},</span> <span class="nx">l</span><span class="o">:</span> <span class="p">{},</span>

    <span class="nx">r</span><span class="o">:</span> <span class="kd">function</span><span class="p">(</span><span class="nx">name</span><span class="p">,</span> <span class="nx">callback</span><span class="p">)</span> <span class="p">{</span> <span class="nx">p</span><span class="p">.</span><span class="nx">f</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span> <span class="o">=</span> <span class="nx">callback</span><span class="p">;</span> <span class="p">},</span>

    <span class="nx">g</span><span class="o">:</span> <span class="kd">function</span><span class="p">(</span><span class="nx">name</span><span class="p">)</span> <span class="p">{</span>
        <span class="k">if</span> <span class="p">(</span><span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">name</span><span class="p">])</span> <span class="p">{</span>
            <span class="k">return</span> <span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">name</span><span class="p">];</span>
        <span class="p">}</span>

        <span class="k">if</span> <span class="p">(</span><span class="nx">p</span><span class="p">.</span><span class="nx">l</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span> <span class="o">||</span> <span class="o">!</span><span class="nx">p</span><span class="p">.</span><span class="nx">f</span><span class="p">[</span><span class="nx">name</span><span class="p">])</span> <span class="p">{</span>
            <span class="k">throw</span> <span class="s2">&quot;p:&quot;</span> <span class="o">+</span> <span class="nx">name</span><span class="p">;</span>
        <span class="p">}</span>

        <span class="nx">p</span><span class="p">.</span><span class="nx">l</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span> <span class="o">=</span> <span class="kc">true</span><span class="p">;</span>

        <span class="k">try</span> <span class="p">{</span>
            <span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span> <span class="o">=</span> <span class="nx">p</span><span class="p">.</span><span class="nx">f</span><span class="p">[</span><span class="nx">name</span><span class="p">](</span><span class="nx">p</span><span class="p">);</span>
        <span class="p">}</span> <span class="k">finally</span> <span class="p">{</span>
            <span class="k">delete</span> <span class="nx">p</span><span class="p">.</span><span class="nx">l</span><span class="p">[</span><span class="nx">name</span><span class="p">];</span>
        <span class="p">}</span>

        <span class="k">return</span> <span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">name</span><span class="p">];</span>
    <span class="p">},</span>
<span class="p">};</span>
</code></pre></div></figure></notextile></div>


<p>That’s 186 bytes once compressed. Not bad! Note that we have twice the same
line in the <code>g</code> function (previously known as “<code>get</code>”):</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='js'><span class="k">return</span> <span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">name</span><span class="p">];</span>
</code></pre></div></figure></notextile></div>


<p>We can invert the first <code>if</code> condition and fit the whole code in it;
combining both <code>return</code>s into one. This is equivalent to transforming this
code:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='js'><span class="kd">function</span> <span class="p">()</span> <span class="p">{</span>
    <span class="k">if</span> <span class="p">(</span><span class="nx">A</span><span class="p">)</span> <span class="p">{</span>
        <span class="k">return</span> <span class="nx">B</span><span class="p">;</span>
    <span class="p">}</span>

    <span class="c1">// ...</span>
    <span class="k">return</span> <span class="nx">B</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></figure></notextile></div>


<p>Into this one:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='js'><span class="kd">function</span> <span class="p">()</span> <span class="p">{</span>
    <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">A</span><span class="p">)</span> <span class="p">{</span>
        <span class="c1">// ...</span>
    <span class="p">}</span>

    <span class="k">return</span> <span class="nx">B</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></figure></notextile></div>


<p>The first form is preferable because it removes one indentation level for the
function body. But here <code>return</code> is a keyword we can’t compress.</p>

<p>Speaking of keyword we can’t compress; how could we remove the <code>delete</code>? All we
care about is to know if there’s a lock or not, so we can set the value to
<code>false</code> instead, at the expense of more memory. This saves us only one byte but
since we only care about the <em>boolean</em> values we can replace <code>true</code> with <code>1</code>
and <code>false</code> with <code>0</code>.</p>

<p>We’re now at 166 bytes and the <code>g</code> function looks like this:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='js'><span class="kd">function</span><span class="p">(</span><span class="nx">name</span><span class="p">)</span> <span class="p">{</span>
    <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">name</span><span class="p">])</span> <span class="p">{</span>
        <span class="k">if</span> <span class="p">(</span><span class="nx">p</span><span class="p">.</span><span class="nx">l</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span> <span class="o">||</span> <span class="o">!</span><span class="nx">p</span><span class="p">.</span><span class="nx">f</span><span class="p">[</span><span class="nx">name</span><span class="p">])</span> <span class="p">{</span>
            <span class="k">throw</span> <span class="s2">&quot;p:&quot;</span> <span class="o">+</span> <span class="nx">name</span><span class="p">;</span>
        <span class="p">}</span>

        <span class="nx">p</span><span class="p">.</span><span class="nx">l</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span> <span class="o">=</span> <span class="mi">1</span><span class="p">;</span>

        <span class="k">try</span> <span class="p">{</span>
            <span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span> <span class="o">=</span> <span class="nx">p</span><span class="p">.</span><span class="nx">f</span><span class="p">[</span><span class="nx">name</span><span class="p">](</span><span class="nx">p</span><span class="p">);</span>
        <span class="p">}</span> <span class="k">finally</span> <span class="p">{</span>
            <span class="nx">p</span><span class="p">.</span><span class="nx">l</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
        <span class="p">}</span>
    <span class="p">}</span>

    <span class="k">return</span> <span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">name</span><span class="p">];</span>
<span class="p">}</span>
</code></pre></div></figure></notextile></div>


<p>Now, what if we tried to remove one of the three objects we’re using? We need
to keep the functions and the results in separate objects but we might be able
to remove the locks object without losing the functionality.</p>

<p>Assuming that modules only return objects let’s merge <code>m</code> and <code>l</code>. We’ll set
<code>p.m[A]</code> to <code>0</code> if it’s locked and will then override the lock with the result.
<code>p.m[A]</code> then have the following possible values:</p>

<ul>
<li><code>undefined</code>: the key doesn’t exist; the module hasn’t been required yet</li>
<li><code>0</code>: the module is currently being executed</li>
<li>something else: the module has already been executed; we have its return
value</li>
</ul>


<p>We need to modify our code a little bit for this:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='js'><span class="kd">function</span><span class="p">(</span><span class="nx">name</span><span class="p">)</span> <span class="p">{</span>
    <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">name</span><span class="p">])</span> <span class="p">{</span>
        <span class="k">if</span> <span class="p">(</span><span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span> <span class="o">===</span> <span class="mi">0</span> <span class="o">||</span> <span class="o">!</span><span class="nx">p</span><span class="p">.</span><span class="nx">f</span><span class="p">[</span><span class="nx">name</span><span class="p">])</span> <span class="p">{</span>
            <span class="k">throw</span> <span class="s2">&quot;p:&quot;</span> <span class="o">+</span> <span class="nx">name</span><span class="p">;</span>
        <span class="p">}</span>

        <span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
        <span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span> <span class="o">=</span> <span class="nx">p</span><span class="p">.</span><span class="nx">f</span><span class="p">[</span><span class="nx">name</span><span class="p">](</span><span class="nx">p</span><span class="p">);</span>
    <span class="p">}</span>

    <span class="k">return</span> <span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">name</span><span class="p">];</span>
<span class="p">}</span>
</code></pre></div></figure></notextile></div>


<p>Note that this allowed us to get ride of the <code>try</code>/<code>finally</code> which let us go
down to 143 bytes. We can already save two bytes by using <code>&lt; 1</code> instead of
<code>=== 0</code>.</p>

<p>Replacing <code>||</code> (boolean <code>OR</code>) with <code>|</code> (binary <code>OR</code>) saves one more byte and
allows us to fit in 140 bytes! We can go further and remove the brackets for
the inner <code>if</code> since it only has one instruction. We need to do that after the
compression because YUI Compressor adds brackets if they’re missing.</p>

<p>The final code looks like this:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='js'><span class="nx">p</span> <span class="o">=</span> <span class="p">{</span>
    <span class="nx">f</span><span class="o">:</span> <span class="p">{},</span> <span class="nx">m</span><span class="o">:</span> <span class="p">{},</span>

    <span class="nx">r</span><span class="o">:</span> <span class="kd">function</span><span class="p">(</span><span class="nx">name</span><span class="p">,</span> <span class="nx">callback</span><span class="p">)</span> <span class="p">{</span> <span class="nx">p</span><span class="p">.</span><span class="nx">f</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span> <span class="o">=</span> <span class="nx">callback</span><span class="p">;</span> <span class="p">},</span>

    <span class="nx">g</span><span class="o">:</span> <span class="kd">function</span><span class="p">(</span><span class="nx">name</span><span class="p">)</span> <span class="p">{</span>
        <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">name</span><span class="p">])</span> <span class="p">{</span>
            <span class="k">if</span> <span class="p">(</span><span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span> <span class="o">&lt;</span> <span class="mi">1</span> <span class="o">|</span> <span class="o">!</span><span class="nx">p</span><span class="p">.</span><span class="nx">f</span><span class="p">[</span><span class="nx">name</span><span class="p">])</span>
                <span class="k">throw</span> <span class="s2">&quot;p:&quot;</span> <span class="o">+</span> <span class="nx">name</span><span class="p">;</span>

            <span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
            <span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span> <span class="o">=</span> <span class="nx">p</span><span class="p">.</span><span class="nx">f</span><span class="p">[</span><span class="nx">name</span><span class="p">](</span><span class="nx">p</span><span class="p">);</span>
        <span class="p">}</span>

        <span class="k">return</span> <span class="nx">p</span><span class="p">.</span><span class="nx">m</span><span class="p">[</span><span class="nx">name</span><span class="p">];</span>
    <span class="p">},</span>
<span class="p">};</span>
</code></pre></div></figure></notextile></div>


<p>That’s 139 bytes once compressed! You can see the result at the top of this
blog post.  <br/>
Please add a comment below if you think of any way to reduce this
further while preserving all existing features.</p>

<p>Thank you for reading!</p>


feed_id: http://bfontaine.net/blog/2015/01/17/preventing-bash-pranks
title: Preventing Bash Pranks
uri: 2015-11-03-Preventing-Bash-Pranks
url: http://bfontaine.net/blog/blog/2015/01/17/preventing-bash-pranks/
content:
<p>The easiest and most popular Bash pranks involve someone messing up with your
<code>~/.bashrc</code>. For example, here is a real-life example:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span class="c">#! /bin/bash</span>
<span class="nv">b</span><span class="o">=</span>~/.bashrc
<span class="nb">echo</span> &gt;&gt;<span class="nv">$b</span>
<span class="nb">echo</span> <span class="s2">&quot;echo sleep 1 &gt;&gt;$b&quot;</span> &gt;&gt;<span class="nv">$b</span>
</code></pre></div></figure></notextile></div>


<p>If you execute this script, it’ll add a newline in your <code>~/.bashrc</code> just in
case it doesn’t end with a newline, then add this line:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span class="nb">echo </span>sleep 1 &gt;&gt;~/.bashrc
</code></pre></div></figure></notextile></div>


<p>The effect of this isn’t immediately visible to the pranked user. When they’ll
start a new Bash session, e.g. by opening a new terminal window, the code in
<code>~/.bashrc</code> will be executed, and the previous line will add <code>sleep 1</code> at the
end of it, which means it’ll be executed and the user will have to wait one
more second before having their prompt. The next time they’ll open a session,
it’ll add one more line and thus will wait 2 seconds, and so forth.</p>

<p>In this post, I’ll give you an overview of the existing solutions to prevent
these pranks.</p>

<!--more-->


<p>Note that I’m referring to <code>~/.bashrc</code> as your startup Bash file because it’s
commonly used, but some people directly use <code>~/.bash_profile</code> instead, or
another one. When you start a session, Bash reads <code>/etc/profile</code>, then tries
<code>~/.bash_profile</code>, <code>~/.bash_login</code>, and <code>~/.profile</code>, (in that order). In most
environments the default <code>~/.bash_profile</code> file sources <code>~/.bashrc</code>.</p>

<h2>User Rights</h2>

<p>The first solution is to protect your <code>~/.bashrc</code> by restraining the access.
Nobody should be able to edit your file except <em>you</em> (and <code>root</code>). It should be
the default, but if you messed up with user rights, here is how to reset the
file to a safe state (read and write for you, and that’s <em>all</em>):</p>

<pre><code>$ chmod 600 ~/.bashrc
</code></pre>

<p>Most attacks thus involve you executing a script, which allows them to bypass
the rights because the script is executed by <em>you</em> with <em>your</em> editing rights.</p>

<p>One solution would be to remove your own writing right and adding it only when
you need it:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span class="c"># add this in your ~/.bashrc</span>
secure-edit<span class="o">()</span> <span class="o">{</span> chmod u+w <span class="nv">$*</span>; <span class="k">${</span><span class="nv">EDITOR</span><span class="k">:-</span><span class="nv">vi</span><span class="k">}</span> <span class="nv">$*</span>; chmod u-w <span class="nv">$*</span>; <span class="o">}</span>
</code></pre></div></figure></notextile></div>


<p>Then remove your writing right:</p>

<pre><code>$ chmod 400 ~/.bashrc
</code></pre>

<p>You can’t edit your file anymore, but you can use your new <code>secure-edit</code>
command:</p>

<pre><code>$ secure-edit ~/.bashrc
</code></pre>

<p>It temporarily allows you to modify the file, open your editor, then put the
restricted rights back.</p>

<h2>The “last line protection”</h2>

<p>This one is easy to use but easy to circumvent. The goal is to prevent one-line
insertions, such as:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span class="nb">echo</span> <span class="s2">&quot;alias ls=cd&quot;</span> &gt;&gt; ~/.bashrc
</code></pre></div></figure></notextile></div>


<p>and the solution is as simple as:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span class="c">#</span>
</code></pre></div></figure></notextile></div>


<p>Yes, that’s just an hash symbol. If you ends your <code>~/.bashrc</code> with it, the
first inserted line will be commented out:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span class="c">#alias ls=cd</span>
</code></pre></div></figure></notextile></div>


<p>It doesn’t work if the prankster adds multiple lines, or adds a newline before
the prank.</p>

<h2><code>return</code></h2>

<p>You can exit from a Bash script with <code>exit</code>. Your <code>~/.bashrc</code> is not executed
like a script, it’s <em>sourced</em>. This means Bash doesn’t start a subshell for it
and execute all its content in the current shell. This also means if you write
<code>exit</code> it’ll exit your current shell.</p>

<p>The solution here is to use <code>return</code> at the end of your file:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span class="k">return</span>
</code></pre></div></figure></notextile></div>


<p>Any line added after this one won’t be executed because Bash will stop the
evaluation. Note that while it’s better than the previous solution, it can be
nullified by a <code>sed</code> call (e.g. <code>sed 's/return//'</code>).</p>

<h2>The disguised <code>return</code></h2>

<p>This one is the same as the previous one, but prevents pranksters from removing
it with calls to <code>sed</code> or similar search &amp; replace techniques. It uses the fact
than in Bash you can execute a command contained in a variable by using it at
the proper place:</p>

<pre><code>print_something=echo
$print_something hello
</code></pre>

<p>These lines are equivalent to <code>echo hello</code>. We use the same thing here with
<code>return</code>. The idea is to execute an obfuscated version of return, e.g.:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span class="nv">b</span><span class="o">=</span>tu
<span class="nv">a</span><span class="o">=</span>re
<span class="nv">c</span><span class="o">=</span>rn

<span class="nv">$a$b$c</span>
</code></pre></div></figure></notextile></div>


<p>And <em>voilà!</em> It’s now nearly impossible to detect the <code>return</code> execution
without manually editing the <code>~/.bashrc</code> file.</p>

<p>This is still vulnerable to file replacement, e.g.:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'>rm ~/.bashrc
<span class="nb">echo</span> <span class="s1">&#39;echo &quot;sorry, no.&quot;&#39;</span> &gt; ~/.bashrc
</code></pre></div></figure></notextile></div>


<p>This wipes the existing <code>~/.bashrc</code> file and replace it with another one.</p>


feed_id: http://bfontaine.net/blog/2014/11/11/a-python-toolbox
title: A Python toolbox
uri: 2015-11-03-A-Python-toolbox
url: http://bfontaine.net/blog/blog/2014/11/11/a-python-toolbox/
content:
<p>I started learning Python four years ago and have been heavily programming with
it for near than a year now. In this post I’ll share some tools I use to ease
and speed-up my workflow, either in the Python code or in the development
environment.</p>

<!-- more -->


<p>These tips should be OS- and editor-independent. I have some useful Vim plugins
to work with Python, but that’ll be for another post. You might have to adapt
commands if you work on Windows.</p>

<h2>Setting Things Up</h2>

<p>Let’s say you’re starting a Python library. You have a couple dependencies, and
hopefully you’d like it to work on multiple versions, like 2.6, 2.7, and 3.x.
How can you test that? You have to (a) manage your dependencies to not mess up
with your user environment, and (b) test with multiple Python versions.</p>

<p>Introducing <a href="http://virtualenv.readthedocs.org/en/latest/virtualenv.html">virtualenv</a>.</p>

<p>Virtualenv lets you create <em>isolated</em> Python environments. That means you get a
pristine environment where you’ll install your library’s dependencies, and
nothing else. It’ll be independent of your user space. It’s important to work
with an isolated environment because you don’t know which environment will your
users have, so you shouldn’t make any assumption besides your own requirements.
Working with your user environment means you might forgot a dependency because
it just works since it’s installed on your computer but it’ll broke if run on
another computer without this dependency.</p>

<p>You should be able to install it with <code>pip</code>:</p>

<pre><code>$ pip install virtualenv
</code></pre>

<p>It needs a directory to store the environment. I usually use <code>venv</code>, but you
can choose whatever you want:</p>

<pre><code>$ virtualenv venv
</code></pre>

<p>You can then either “activate” the environment, which adds its <code>bin</code> directory
in your <code>PATH</code>:</p>

<pre><code>$ source venv/bin/activate
$ python  # that's your virtualenv's Python
$ deactivate
$ python  # that's your Python
</code></pre>

<p>or prefix your commands with <code>venv/bin/</code> (replace <code>venv</code> with your directory’s
name):</p>

<pre><code>$ venv/bin/python  # that's your virtualenv's Python
$ python  # that's your Python
</code></pre>

<p>I usually do the later. Install dependencies in the environment:</p>

<pre><code>$ venv/bin/pip install your-dependency
</code></pre>

<p>Don’t forget to tell Git or any tool you’re using for versioning to ignore
this <code>venv</code> directory. It can take some place (from 12MB to more than 40MB)
depending on the number of dependencies you’re relying on.</p>

<p>To remove an environment, just delete its directory:</p>

<pre><code>$ rm -rf venv
</code></pre>

<p>It can be convenient to save place on your computer if you have dozen of
environments for different projects, especially if can quickly re-create any
environment with its dependencies.</p>

<p>If you’re using <code>pip</code> to manage them, you should know you can install
dependencies not only from the command line, but also from a file, with one
dependency per line. Each one of them is processed as if it were given on the
command-line.</p>

<p>For example, you could have <a href="https://github.com/bfontaine/term2048/blob/master/requirements.txt">a file</a> containing this:</p>

<pre><code>colorama==0.2.7
coverage==3.7.1
pep8==1.4.6
py==1.4.20
tox==1.7.0
argparse&gt;=1.1
virtualenv==1.11.4
</code></pre>

<p>This file is usually called <code>requirements.txt</code>, but here again you can call it
whatever you want. You call install all these at once with this command:</p>

<pre><code>$ pip install -r requirements.txt
</code></pre>

<p>But we’re programmers and we’re lazy, we don’t want to track down each
installed library to include it in this file.</p>

<p>Here comes <code>pip freeze</code>.</p>

<p><code>pip freeze</code> outputs all installed libraries with their version. It can be put
in our <code>requirements.txt</code> for later use:</p>

<pre><code>$ pip freeze &gt; requirements.txt
</code></pre>

<p>This <code>requirements.txt</code> file becomes handy when used with <code>virtualenv</code> because
you’re now able to fire up a new environment and install all required
libraries, with two commands:</p>

<pre><code>$ virtualenv venv
$ venv/bin/pip install -r requirements.txt
</code></pre>

<p>Note that these are libraries used in the environment, not necessarily your
library’s dependencies. In the above example you can see we’re installing
<code>coverage</code> and <code>pep8</code>, which respectively are a code coverage test tool and a
lint one we’ll talk about later in this post, not libraries we’re depending on
here.</p>

<p>You should thus add this file in your public repository, because it provides
anyone with the informations they need to have in order to mirror your
environment and be able to contribute to your project.</p>

<h2>Coding</h2>

<p>Now that your local environment is ready, you can start coding your library.
You’ll often have to fire an interpreter to test some things, use <code>help()</code> to
check a function’s arguments, etc. Having to type the same things over and over
takes time, and remember: we’re lazy.</p>

<p>Like Bash and some other tools, the Python interpreter can be configured with
an user file, namely <code>$PYTHONSTARTUP</code>. It allows you to add autocompletion,
import common modules, and execute pretty much any code you want.</p>

<p>Start by setting <code>PYTHONSTARTUP</code> in your <code>~/.bashrc</code> (if you’re using Bash):</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span class="nb">export </span><span class="nv">PYTHONSTARTUP</span><span class="o">=</span><span class="s2">&quot;$HOME/.pythonrc.py&quot;</span>
</code></pre></div></figure></notextile></div>


<p>Here we’re telling the interpreter to look for the <code>$HOME/.pythonrc.py</code> file
and executing it before giving us a prompt.</p>

<p>Let’s initialize this file with autocompletion support:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='python'><span class="k">try</span><span class="p">:</span>
    <span class="kn">import</span> <span class="nn">readline</span>
<span class="k">except</span> <span class="ne">ImportError</span><span class="p">:</span>
    <span class="k">pass</span>
<span class="k">else</span><span class="p">:</span>
    <span class="kn">import</span> <span class="nn">rlcompleter</span>
    <span class="n">readline</span><span class="o">.</span><span class="n">parse_and_bind</span><span class="p">(</span><span class="s">&quot;tab: complete&quot;</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>


<p>You can add a lot more <a href="http://www.sontek.net/blog/2010/12/28/tips_and_tricks_for_the_python_interpreter.html">stuff</a> in this file, like <a href="https://github.com/bfontaine/Dotfiles/blob/master/.pythonrc.py#L21-L38">history</a>
support, <a href="https://github.com/coderholic/config/blob/master/.pythonrc.py#L77-L80">colored prompts</a> or common imports. For example, if you use <code>sys</code>
and <code>re</code> a lot, you can save time by adding them in your startup file:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='python'><span class="kn">import</span> <span class="nn">sys</span>
<span class="kn">import</span> <span class="nn">re</span>
</code></pre></div></figure></notextile></div>


<p>You won’t need to type these two lines anymore in your interpreter. It doesn’t
change how Python executes files, just the interactive interpreter.</p>

<h2>Testing</h2>

<p>Four different kinds of tests are covered by this part: style checkers to
ensure your code’s consistency, static analysis tools to detect problems before
executing the code, unit tests to actually test your library, and code coverage
tests to ensure you do test <em>all</em> the code.</p>

<h3>Style Checking</h3>

<p>These are tools which help you maintaining a consistent coding style in your
whole codebase. Most of these tools are easy to use, the hardest part being to
choose which one fits your requirements.</p>

<p>Python has a PEP (<q>Python Enhancement Proposal</q>, sort of RFC), the
<a href="http://legacy.python.org/dev/peps/pep-0008/">PEP&nbsp;8</a>, dedicated to its coding conventions. If you want to follow
it, a command-line tool, rightly named <code>pep8</code>, is available.</p>

<pre><code>$ venv/bin/pip install pep8
$ venv/bin/pep8 your/lib/root/directory
...
foo/mod.py:84:80: E501 line too long (96 &gt; 79 characters)
foo/mod.py:85:6: E203 whitespace before ':'
foo/mod.py:86:80: E501 line too long (87 &gt; 79 characters)
foo/mod.py:87:4: E121 continuation line indentation is not a multiple of four
...
</code></pre>

<p>It’ll check each file and print a list of warnings. You can choose to hide some
of them, or use a white list to decide which ones you want. It’s a good tool if
you want to follow the PEP&nbsp;8 conventions.</p>

<p>Another highly customizable tool is <code>pylint</code>. It reads its configuration from
a file in your project, which can <a href="http://stackoverflow.com/a/22449845/735926">inherit</a> from global and user
configurations. It’ll warn you about bad naming, missing docstrings, functions
which take too many arguments, duplicated code, etc. It also gives you some
statistics about your code. It’s really powerful but can be a pain if you don’t
configure it. For example, it warns you about one-letter variables while you
might find them ok.</p>

<p>Enters <code>prospector</code>.</p>

<p>Prospector is built on top of <code>pep8</code> and <code>pylint</code> and comes with sane defaults
regarding the pickiness of both tools. You can <a href="http://blog.landscape.io/prospector-python-static-analysis-for-humans.html">tell it</a> to only
print important problems about your code:</p>

<pre><code>$ venv/bin/pip install prospector
$ venv/bin/prospector --strictness high
</code></pre>

<p>You’ll get a much shorter output, which will hopefully help you find potential
problems in your code.</p>

<h3>Static Analysis</h3>

<p>Here, we’re talking about analysing the code without executing it. Compiled
languages benefit from this at compilation time, but interpreted languages like
Python have no way to have it.</p>

<p>One of the most popular tools for static analysis on Python code is
<a href="https://launchpad.net/pyflakes">Pyflakes</a>. It doesn’t check your coding style like <code>pep8</code> or <code>pylint</code>, but
warns you about missing imports, dead code, unused variables, redefined
functions and more. You can work without style checkers, but static analysis is
really helpful to detect potential bugs before actually running the code.</p>

<p>Pyflakes can be integrated in editors like Vim with <a href="https://github.com/scrooloose/syntastic#readme">Syntastic</a>, but its
command-line usage is as easy as the previous tools:</p>

<pre><code>$ venv/bin/pip install pyflakes
$ venv/bin/pyflakes your/directory
</code></pre>

<p>Prospector, mentioned in the previous section, also includes <code>pyflakes</code>. You
might also want to try <a href="https://pypi.python.org/pypi/flake8">Flake8</a>, which combines <code>pyflakes</code> and <code>pep8</code>.</p>

<h3>Unit Tests</h3>

<p>When talking about <q>testing</q>, we usually think of <em>unit</em> testing, which is
testing small pieces of our code at a time, to make sure everything is working
correctly. The goal is to test only one feature at a time, to quickly be able
to find which parts of the code are not working. There are a lot of great
testing frameworks, and Python comes with a built-in one, <a href="https://docs.python.org/2/library/unittest.html">unittest</a>, which I
personnally use. I won’t cover these, and I’ll instead cover the case when you
need to test on multiple Python versions, which is often the case when you
plan to release a public library. You obviously don’t want to manually switch
to each Python version, install your dependencies then run your tests suit each
time.</p>

<p>This is a job for <code>tox</code>.</p>

<p><a href="https://testrun.org/tox/latest/">Tox</a> uses Virtualenv, which I talked about earlier, to create standalone
Python environments for different Python versions, and test your code in each
one of them.</p>

<pre><code>$ venv/bin/pip install tox
</code></pre>

<p>Like some previous tools, it needs a configuration file. Here is a basic
<code>tox.ini</code> to test on Python 2.6, 2.7, 3.3 and 3.4:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='ini'><span class="k">[tox]</span>
<span class="na">envlist</span> <span class="o">=</span> <span class="s">py26, py27, py33, py34</span>
<span class="na">downloadcache</span> <span class="o">=</span> <span class="s">{toxworkdir}/_download/</span>

<span class="k">[testenv]</span>
<span class="na">sitepackages</span> <span class="o">=</span> <span class="s">False</span>
<span class="na">deps</span> <span class="o">=</span><span class="s"></span>
<span class="s">  colorama</span>
<span class="na">commands</span> <span class="o">=</span><span class="s"></span>
<span class="s">  {envpython} {toxinidir}/tests/test.py</span>
</code></pre></div></figure></notextile></div>


<p>It declares one dependency, <code>colorama</code>, and tells <code>tox</code> to run tests by
executing <code>tests/test.py</code>. That’s all. We can then run our tests:</p>

<pre><code>$ venv/bin/pip/tox
</code></pre>

<p>It’ll takes some time on the first run to fetch dependencies and create
environments, but all the following times will be faster.</p>

<p>Like <code>virtualenv</code>, <code>tox</code> uses a directory to store these environments. You can
safely delete it if you need more space, it’ll be re-created by <code>tox</code> the next
time:</p>

<pre><code>$ rm -rf .tox
</code></pre>

<h3>Code Coverage</h3>

<p>This last part about testing talks about code coverage tests. These are tests
about tests. The goal here is to ensure your tests cover all your code, and
you don’t leave some parts untested. Most tools tell you how many lines where
executed when running your tests suit, and give you an overall coverage
percentage.</p>

<p>One of them is <code>coverage</code>.</p>

<pre><code>$ venv/bin/pip install coverage
</code></pre>

<p>Give it your project’s root directory as well as a file to run your tests:</p>

<pre><code>$ venv/bin/coverage run --source=your/directory tests/test.py
</code></pre>

<p>It’ll run them, and give you a nice coverage report:</p>

<pre><code>$ coverage report -m
Name              Stmts   Miss  Cover   Missing
-----------------------------------------------
foobar/__init__       5      0   100%
foobar/base          46      2    96%   5-6
foobar/cli          159    159     0%   3-280
foobar/config        66     66     0%   3-150
foobar/session       64     14    78%   9-10, 106-124
foobar/barfooqux     22      0   100%
foobar/helloworld    25     20    20%   14-41
...
-----------------------------------------------
TOTAL               459    313    32%
</code></pre>

<p>It can also give you an HTML version:</p>

<pre><code>$ venv/bin/coverage html
</code></pre>

<p>Getting to 100% is the ultimate goal, but you’ll quickly find that’s the first
80% are easy and the remaining 20% are the hardest part, especially when you
have I/O, external dependencies like databases, and/or complicated
corner-cases.</p>

<p>Check <a href="http://nedbatchelder.com/code/coverage/">Coverage</a>’s doc for more info.</p>

<h2>Debugging</h2>

<p>There are a lot of ways to debug, including logs, but the simplest debugging
tool is the good old <code>print</code>. It becomes really impracticable when you have to
restart your server every time you add or remove one of them from your code.
What if you could fire a Python interpreter right from your code and inspect it
when it’s running? Well, you can do that with Python’s <code>code</code> module! This
trick is really handy, and I’ve been using it heavily instead of these <code>print</code>s
we write everywhere since I’ve discovered it.</p>

<p>The <code>code</code> module provides you with an <code>interact</code> function, which takes a
<code>dict</code> of variables to inject in the interpreter. You’ll be able to print them,
play with them, but these changes won’t be reflected in the program you’re
debugging.</p>

<p>Remember that Python lets you get all global variables as a <code>dict</code> with
<code>globals()</code> and all local ones with <code>locals()</code>. We thus start by creating a
mirror of the local environment:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='python'><span class="nb">vars</span> <span class="o">=</span> <span class="nb">globals</span><span class="p">()</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
<span class="nb">vars</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="nb">locals</span><span class="p">())</span>
</code></pre></div></figure></notextile></div>


<p>These two lines get all global variables (including imported modules) in a
<code>dict</code> called <code>vars</code> and add local variables in it. This can then be passed
directly to the interpreter:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='python'><span class="kn">import</span> <span class="nn">code</span>
<span class="n">code</span><span class="o">.</span><span class="n">interact</span><span class="p">(</span><span class="n">local</span><span class="o">=</span><span class="nb">vars</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>


<p>This will start an interpreter with all these variables already available in
it. There’s nothing to install, this is a <a href="https://docs.python.org/2/library/code.html#code.interact">standard Python module</a>.</p>

<p>You can even inline the code and add it in your favorite snippets manager:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='python'><span class="n">g</span><span class="o">=</span><span class="nb">globals</span><span class="p">()</span><span class="o">.</span><span class="n">copy</span><span class="p">();</span><span class="n">g</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="nb">locals</span><span class="p">());</span><span class="kn">import</span> <span class="nn">code</span><span class="p">;</span><span class="n">code</span><span class="o">.</span><span class="n">interact</span><span class="p">(</span><span class="n">local</span><span class="o">=</span><span class="n">g</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>


<p>Don’t forget to remove it when you’re done!</p>

<h2>TL;DR</h2>

<ul>
<li>Use <code>virtualenv</code> to isolate your Python environment, <code>pip freeze</code> and a
<code>requirements.txt</code> file to keep track of your dependencies.</li>
<li>Write a <code>pythonrc.py</code> file to add autocomplete support to your interpreter</li>
<li>Use <code>pep8</code>, <code>pylint</code> and <code>pyflakes</code> to keep a high code quality</li>
<li>Use <code>tox</code> to test on multiple Python versions</li>
<li>Fire a local interpreter instead of <code>print</code>ing variables</li>
</ul>


<p>That was all. Please comment on this post if you think of any tool you use to
speed-up your Python development!</p>


title: How to remember the difference between conj and cons in Clojure
uri: 2015-11-03-How-to-remember-the-difference-between-conj-and-cons-in-Clojure
feed_id:
http://bfontaine.net/blog/2014/05/25/how-to-remember-the-difference-between-conj-and-cons-in-clojure

url:
http://bfontaine.net/blog/blog/2014/05/25/how-to-remember-the-difference-between-conj-and-cons-in-clojure/

content:
<p>When I started writing Clojure, I couldn’t memorize the difference between
<code>conj</code> and <code>cons</code> and always used one instead of the another. Their name are
similar, but <code>cons</code> is used to add an element at the beginning of a vector,
while <code>conj</code> is used to add an element at the end of it. How can one memorize
this? I found a mnemonic trick over the time that helps me remember this. Here
is it:</p>

<!-- more -->


<p>The trick is to look at the last letter of each function, <code>s</code> and <code>j</code>. As shown
in the image below, the <code>s</code> of <code>cons</code> shows the right, while the <code>j</code> of
<code>conj</code> shows the left.</p>

<p><img class="center" src="media/a657f802.sj.png" data-original-source="http://bfontaine.net/blog/images/sj.png"></p>

<p>This means that <code>cons</code> pushes elements from the left to the right, that is,
<strong>at the beginning</strong> of a vector. <code>conj</code>, on the other hand, pushes elements
from the right to the left, which is <strong>at the end</strong> of a vector. That’s it.
Once you see this in your head, you’ll never forget the difference between
<code>cons</code> and <code>conj</code> on a vector.</p>


feed_id: http://bfontaine.net/blog/2014/03/03/fixing-gnuplot-pys-unknown-aqua-terminal-warning-on-osx
title: Fixing gnuplot-py’s “unknown aqua terminal” warning on OSX
uri: 2015-11-03-Fixing-gnuplot-py-s-unknown-aqua-terminal-warning-on-OSX
url: http://bfontaine.net/blog/blog/2014/03/03/fixing-gnuplot-pys-unknown-aqua-terminal-warning-on-osx/
content:
<p>When plotting with <a href="https://pypi.python.org/pypi/gnuplot-py"><code>gnuplot-py</code></a> on OSX, I got an annoying warning
saying that terminal <code>aqua</code> is <q>unknown or ambiguous</q>, <em>even</em> when I use a
different terminal (e.g. <code>postscript</code>). This terminal doesn’t exist on my
Gnuplot installation (4.6.3). In fact, <code>gnuplot-py</code> <a href="https://github.com/jrk/gnuplot-py/blob/96b7c7ec81daa560325f4568393e300de020c353/gp.py#L28-39">uses</a> slightly
different files depending on your platform. OSX’s one is exactly the same as
other UNIX-flavored OSes but its default terminal is <code>aqua</code>. There are two ways
to fix the warning, a hacky one I used before this blog post, and a clean one I
discovered while writting this post. Hope this help!</p>

<!-- more -->


<h2>The hacky way</h2>

<p>Since OSX’s Gnuplot file is the same as Linux’s one apart from its default
term, we just need to tell <code>gnuplot-py</code> we’re on Linux:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='py'><span class="kn">import</span> <span class="nn">sys</span>
<span class="n">_platform</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span> <span class="c"># save the normal sys.platform value</span>

<span class="k">if</span> <span class="n">_platform</span> <span class="o">==</span> <span class="s">&#39;darwin&#39;</span><span class="p">:</span>
    <span class="n">sys</span><span class="o">.</span><span class="n">platform</span> <span class="o">=</span> <span class="s">&#39;linux&#39;</span> <span class="c"># replace it with &#39;linux&#39;</span>

<span class="kn">import</span> <span class="nn">Gnuplot</span><span class="o">,</span> <span class="nn">Gnuplot.funcutils</span> <span class="c"># import Gnuplot</span>
<span class="n">sys</span><span class="o">.</span><span class="n">platform</span> <span class="o">=</span> <span class="n">_platform</span> <span class="c"># restore it</span>
</code></pre></div></figure></notextile></div>


<p>It temporarily changes <code>sys.platform</code> value, which <code>gnuplot-py</code> <a href="https://github.com/jrk/gnuplot-py/blob/96b7c7ec81daa560325f4568393e300de020c353/gp.py#L28-39">is relying on</a>,
to <code>"linux"</code>, import Gnuplot and then restore it back to its previous value
(<code>"darwin"</code>).</p>

<h2>The clean way</h2>

<p>In fact there’s a much cleaner way to fix the warning. The default terminal is
stored in <a href="https://github.com/jrk/gnuplot-py/blob/96b7c7ec81daa560325f4568393e300de020c353/gp_unix.py#L86-91"><code>GnuplotOpts.default_term</code></a>, so we just need to change
it to another value to fix the warning:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='py'><span class="kn">import</span> <span class="nn">Gnuplot</span><span class="o">,</span> <span class="nn">Gnuplot.funcutils</span>
<span class="n">Gnuplot</span><span class="o">.</span><span class="n">GnuplotOpts</span><span class="o">.</span><span class="n">default_term</span> <span class="o">=</span> <span class="s">&#39;x11&#39;</span>
</code></pre></div></figure></notextile></div>


<p>It’s cleaner than the hacky way because:</p>

<ol>
<li>It’s clear: the second line is pretty explicit: we change <code>default_term</code> to
<code>'x11'</code>. We don’t even need to add a comment.</li>
<li>It’s more maintainable: if OSX’s <code>gnuplot-py</code> interface change in the
future, this piece of code won’t break compatibility while the hacky way
will prevent our code to use the new interface.</li>
<li>It’s shorter: one line versus five ones. It’s really easier to introduce
bugs when using five lines instead of just one.</li>
</ol>


feed_id: http://bfontaine.net/blog/2014/02/15/using-coveralls-with-clojure
title: Using Coveralls with Clojure
uri: 2015-11-03-Using-Coveralls-with-Clojure
url: http://bfontaine.net/blog/blog/2014/02/15/using-coveralls-with-clojure/
content:
<p><a href="https://coveralls.io/"><strong>Coveralls</strong></a> is a service that keep track of your tests coverage
for you. It can notifies you when your coverage decreases under a custom
threshold, and their bot comments on pull requests to report their tests
coverage. Like Travis-CI, it allows you to add a badge to your readme with
an up-to-date tests coverage percentage.</p>

<p>If you already test your GitHub projects with a CI server like Travis, it’s
very easy to add Coveralls to your workflow. Unfortunately, they have a library
for Ruby, a couple user-provided libraries for other languages such as PHP,
Java and Python, but nothing for Clojure. Fortunately, they provide an API for
unsupported languages like Clojure. Here is how to use it.</p>

<!-- more -->


<h2>Cloverage</h2>

<p><a href="https://github.com/lshift/cloverage"><strong>Cloverage</strong></a> is a Clojure library and a Leiningen plugin to get
form-level tests coverage of your projects. It’s dead easy to use and generates
HTML reports for you as well as some other formats.</p>

<p>Add their plugin to your <code>project.clj</code>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='clj'><span class="p">(</span><span class="kd">defproject </span><span class="nv">your-project</span> <span class="s">&quot;0.1.0&quot;</span>
  <span class="c1">; ...</span>
  <span class="ss">:plugins</span> <span class="p">[[</span><span class="nv">lein-cloverage</span> <span class="s">&quot;1.0.2&quot;</span><span class="p">]])</span>
</code></pre></div></figure></notextile></div>


<p>Use <code>lein cloverage</code> to run your tests and generate various reports. You can
specify which Cloverage version you want to use with <code>CLOVERAGE_VERSION</code>. The
latest stable one is <code>1.0.3</code>, but we’ll need to use <code>1.0.4-SNAPSHOT</code> for this
article:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='sh'><span class="nv">CLOVERAGE_VERSION</span><span class="o">=</span>1.0.4-SNAPSHOT lein cloverage
</code></pre></div></figure></notextile></div>


<h2>Coveralls</h2>

<p>If you don’t already use it, <a href="https://coveralls.io/authorize/github">signup</a> on Coveralls using your GitHub account.
Then, click on “Add Repo”, find your repo in the list and activate it. That’s
all you have to do on their website for now.</p>

<h2>Travis CI</h2>

<p>You now need to custom your builds on Travis to use Coveralls. We’ll do it in a
Bash script, so add this to your <code>.travis.yml</code>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='yaml'><span class="l-Scalar-Plain">after_script</span><span class="p-Indicator">:</span>
  <span class="p-Indicator">-</span> <span class="l-Scalar-Plain">bash -ex test/coveralls.sh</span>
</code></pre></div></figure></notextile></div>


<p>This tells Travis to run <code>test/coveralls.sh</code> after each build. We use Bash
options <code>-e</code> and <code>-x</code> to respectively stop the script at the first failure and
print each command.</p>

<p>Now edit <code>test/coveralls.sh</code> and add the following content:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span class="nv">COVERALLS_URL</span><span class="o">=</span><span class="s1">&#39;https://coveralls.io/api/v1/jobs&#39;</span>
<span class="nv">CLOVERAGE_VERSION</span><span class="o">=</span><span class="s1">&#39;1.0.4-SNAPSHOT&#39;</span> lein2 cloverage -o cov --coveralls
curl -F <span class="s1">&#39;json_file=@cov/coveralls.json&#39;</span> <span class="s2">&quot;$COVERALLS_URL&quot;</span>
</code></pre></div></figure></notextile></div>


<p>It’ll run Cloverage and output reports in <code>cov</code> (use another name if you
prefer), using a report for Coveralls. This generates a <code>coveralls.json</code> file
which can then be sent to their API. That’s what we do on the next line, using
cURL.</p>

<p>That’s all! You can now push on GitHub, Travis will run your tests and send
their tests coverage to Coveralls.</p>

<h2>Caveats</h2>

<p>Coveralls doesn’t support partial-line tests coverage, so we’re cheating a
little bit here using hits count. A line that it never covered has 0 hits, one
that is partially covered has one hit, and a fully covered one has two hits.
There’s <a href="https://github.com/lemurheavy/coveralls-public/issues/216">an open issue</a> regarding this.</p>


feed_id: http://bfontaine.net/blog/2013/10/22/local-vimrcs
title: Local .vimrc's
uri: 2015-11-03-Local-vimrc-s
url: http://bfontaine.net/blog/blog/2013/10/22/local-vimrcs/
content:
<p>I <a href="http://damien.lespiau.name/blog/2009/03/18/per-project-vimrc/">discovered</a> a great feature in Vim today. While we generally
use the same Vim settings everywhere, we sometimes (have to) contribute to
projects with a coding style different of ours, and because nobody wants to
edit their <code>.vimrc</code> everytime to switch between configurations, Vim allows you
to have per project <code>.vimrc</code>&rsquo;s.</p>

<!--more-->


<p>This feature can be enabled with one line in your <code>.vimrc</code>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='vim'><span class="k">set</span> <span class="nb">exrc</span>
</code></pre></div></figure></notextile></div>


<p>When you start Vim, it begins by looking at some predefined locations for a
configuration file (the first one is <code>$HOME/.vimrc</code>).  The <code>exrc</code> option,
disabled by default, tell to Vim to not only load your personal <code>.vimrc</code>, but
also to look in the current directory to check if there’s another <code>.vimrc</code> file,
which will be loaded <em>after</em> your personal one. You can then use it to override
your settings and/or add specific ones for this directory.</p>

<p>Additionally, if you share the local <code>.vimrc</code> with others (e.g. in a Git
repository), you may want to set the <code>secure</code> option at the end of your personal
<code>.vimrc</code>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='vim'><span class="k">set</span> <span class="nb">secure</span>
</code></pre></div></figure></notextile></div>


<p>This disallow shell and write commands in the <code>.vimrc</code> of the local directory.
On <span class="sc">unix</span>-like OSes, this option is set by default if
you’re not the owner of the file.</p>

<p>For more Vim gems, check my <code>.vimrc</code> on <a href="https://github.com/bfontaine/Dotfiles/blob/master/.vimrc">GitHub</a>!</p>


feed_id: http://bfontaine.net/blog/2013/08/26/use-a-custom-tld-for-local-development
title: Use a Custom Tld for Local Development
uri: 2015-11-03-Use-a-Custom-Tld-for-Local-Development
url: http://bfontaine.net/blog/blog/2013/08/26/use-a-custom-tld-for-local-development/
content:
<p>In this post, we’ll create a custom <a href="https://en.wikipedia.org/wiki/Top-level_domain">TLD</a> for local development, and configure
Apache to work with that. It’ll allow you to work on your local version of
<code>mysuperwebsite.com</code> with the local domain <code>mysuperwebsite.dev</code>, with the exact
same URLs, except that little <code>.com</code> which is replaced by <code>.dev</code>.</p>

<!--more-->


<p>This post is focused on OS X, but the tools used are available on Ubuntu and
others. Additionally, you can choose whatever unexisting domain you want, just
replace <code>.dev</code> with the one you chose in the commands described in the article.</p>

<h2>DNSMasq</h2>

<p>The major problem we have when we develop locally with custom domains is that
we have to add an entry to <code>/etc/hosts</code> for <em>each</em> website. Wildcards are not
supported, so you can’t write the following line in it:</p>

<pre><code>127.0.0.1 *.dev
</code></pre>

<p>There are different workarounds, but here we’ll use DNSMasq as
a local DNS resolver. If you don’t have <a href="http://brew.sh/">homebrew</a>, install it
first, then install DNSMasq:</p>

<pre><code>brew install dnsmasq
# enable the daemon on startup
sudo cp $(brew --prefix dnsmasq)/homebrew.mxcl.dnsmasq.plist /Library/LaunchDaemons/
</code></pre>

<p>DNSMasq will run locally and redirect any query for a <code>*.dev</code> domain to the local
host, <code>127.0.0.1</code>. Open <code>/usr/local/etc/dnsmasq.conf</code> (or create it if it doesn’t
exist) and add the following lines:</p>

<pre><code>address=/dev/127.0.0.1
listen-adress=127.0.0.1
</code></pre>

<p>Then start DNSMasq:</p>

<pre><code>sudo launchctl load homebrew.mxcl.dnsmasq
</code></pre>

<p>You’ll have to tell OS X to send its
DNS queries to the local server first, then try the other ones, to intercept
the queries for the <code>*.dev</code> domains. Go to <i>System Preferences → Network →
Advanced → DNS</i>, and add <code>127.0.0.1</code> at the top of the list of DNS servers.</p>

<p>You can then check that it works using <code>dig</code>:</p>

<pre><code>$ dig foobar.dev
…
;; QUESTION SECTION:
;foobar.dev.            IN  A

;; ANSWER SECTION:
foobar.dev.     0   IN  A   127.0.0.1

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
…
</code></pre>

<p>If you have some
server listening on port 80, you can try <code>foobar.dev</code> in your browser, it’ll
display whatever you’re serving on <code>127.0.0.1:80</code>. If you have any troubles,
empty your cache (use <code>dscacheutils -flushcache</code> on OS X), or restart your
computer.</p>

<p>That’s all! You can stop there, but if you’re using Apache you may be
interested by the next section of this article.</p>

<h2><code>mod_vhost_alias</code></h2>

<p>This Apache module allows you manage virtual hosts dynamically, so you won’t
have to create a new one for every (local) website. This is useful if you have
a large number of virtual hosts with similar configurations. In this section,
we’ll see how to associate a <code>.dev</code> domain with a directory on your computer.
We’ll assume you already have Apache installed and working.</p>

<p>With <code>mod_vhost_alias</code>, Apache extracts the hostname from the client query (with
the <code>Host</code> HTTP parameter) and use it to get the directory path. The <a href="https://httpd.apache.org/docs/current/mod/mod_vhost_alias.html">official doc</a>
has a lot of examples, I personally prefer to be able to use whatever directory
name I want, so I’m using a <code>vhosts</code> directory which contains symbolic links to
the right directories. Like with DNSMasq before, you need to add only two lines
of configuration here. Open <code>/etc/apache2/extra/httpd-vhosts.conf</code> (you’ll need
to use <code>sudo</code>), and add the following lines at the end of it:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='apache'><span class="nb">UseCanonicalName</span> <span class="k">Off</span>
<span class="nb">VirtualDocumentRoot</span> <span class="s2">&quot;/Users/baptiste/some/dirs/vhosts/%-2.0.%-1.0&quot;</span>
</code></pre></div></figure></notextile></div>


<p>You may want to customize the path. This one takes the last two parts of the
domain (i.e. the domain and the tld), and use the directory
<code>~/some/dirs/vhosts/domain.tld/</code> for it. Note that it’ll use the same directory
for <code>bar.dev</code>, <code>foo.bar.dev</code>, <code>qux.foo.bar.dev</code>, etc.</p>

<p>Then, use <code>ln -s</code> to make symbolic links from <code>vhosts/</code> to the right directories,
e.g.:</p>

<pre><code>~/some/dirs/vhosts/foo.dev -&gt; ~/sites/foo.dev
~/some/dirs/vhosts/bar.dev -&gt; ~/perso/blog
</code></pre>

<p>If all the websites are in the same directory, you can skip the
“symbolic links” part. Restart Apache with <code>sudo apachectl restart</code> (on Ubuntu,
use <code>sudo service apache2 restart</code>), and you’re done. In the future, you won’t
have to restart Apache for each new site, you need to restart it only when you
modify its configuration.</p>

<p>If you get some issues with rewrite rules, add</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='apache'><span class="nb">RewriteBase</span> /
</code></pre></div></figure></notextile></div>


<p>in the <code>.htaccess</code> of the websites that use them, and it’ll work.</p>


feed_id: http://bfontaine.net/blog/2013/06/04/get-the-urban-dictionary-in-your-terminal
title: Get the Urban Dictionary in your terminal
uri: 2015-11-03-Get-the-Urban-Dictionary-in-your-terminal
url: http://bfontaine.net/blog/blog/2013/06/04/get-the-urban-dictionary-in-your-terminal/
content:
<p>The <a href="http://www.urbandictionary.com/">Urban Dictionary</a> website is pretty useful when one wants to
know the meaning of any familiar or slang word. Users can write definitions for
any words, and upvote or downvote them. Unfortunately, the website doesn’t have
a public API, and I couldn’t find a command-line tool for it. So I wrote one.</p>

<!--more-->


<h2>Introducing ud</h2>

<p><strong>ud</strong> is a command-line tool which prints definitions from the Urban Dictionary
website. It’s <a href="https://github.com/bfontaine/ud">written</a> in Ruby, and allows you to quickly search for a
definition of any word.</p>

<pre><code>$ ud wth
* WTH (567/126):

   1) Abbreviation for "What the Hell"
   2) Shortened alternative to "With"

 Example:
   1) WTH was that?!
   2) I ate some crackers wth chicken
(…)
</code></pre>

<p>For common abbreviations like “wtf” or “omg”, there are dozens of definitions,
with minor differences. That’s why the script scraps only the first page of
results, and supports a few options: <code>-n</code> allows you to limit the number of
definitions (e.g. <code>-n 1</code> → only one). <code>-r</code> allows you to select only the
definitions which have an upvotes/downvotes ratio higher than what you want,
e.g. <code>-r 2</code> will select only the definitions which have twice more upvotes than
downvotes. For your convenience, it also support the shortcut <code>-u</code> which is an
alias for <code>-r 2</code>. Type <code>ud -h</code> for more info.</p>

<h3>Install</h3>

<p>That’s easy:</p>

<pre><code>gem install ud
</code></pre>

<p>Please note that if you’re using Windows, you’ll need to install the Win32
Console ANSI gem for the colored output. Install it with:</p>

<pre><code>gem install win32console
</code></pre>

<h3>Using the UD module</h3>

<p>The gem also provide a <code>UD</code> module, which you can use to query the website in
your own Ruby code:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='ruby'><span class="nb">require</span> <span class="s1">&#39;ud&#39;</span>

<span class="n">wtf</span> <span class="o">=</span> <span class="no">UD</span><span class="o">.</span><span class="n">query</span> <span class="s1">&#39;wtf&#39;</span><span class="p">,</span> <span class="ss">:count</span> <span class="o">=&gt;</span> <span class="mi">1</span>

<span class="nb">puts</span> <span class="n">wtf</span><span class="o">[</span><span class="mi">0</span><span class="o">].</span><span class="n">keys</span><span class="o">.</span><span class="n">join</span> <span class="s1">&#39;, &#39;</span>
<span class="c1">#=&gt; id, word, definition, example, upvotes, downvotes</span>
</code></pre></div></figure></notextile></div>


feed_id: http://bfontaine.net/blog/2016/04/02/syntax-quiz-number-1-rubys-mysterious-percent-suite
title: Syntax Quiz #1: Ruby’s mysterious percent suite
uri: Syntax-Quiz-1-Ruby-s-mysterious-percent-suite
url: http://bfontaine.net/blog/2016/04/02/syntax-quiz-number-1-rubys-mysterious-percent-suite/
content:
<p>This is the first post of a serie I’m starting about syntax quirks in various
languages. I’ll divide each post in two parts: the first one states the
question (mainly “Is this valid? What does it do? How does it work?”); the
second one gives an answer.</p>

<p><strong>In Ruby, any sequence of 3+4n (where n≥0) percent signs (<code>%</code>) is valid;
can you guess why?</strong></p>

<p>Here are the first members of this suite:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='text'>%%%             # n=0
%%%%%%%         # n=1
%%%%%%%%%%%     # n=2
%%%%%%%%%%%%%%% # n=3
</code></pre></div></figure></notextile></div>


<p>I’m using Ruby 2.3.0 but this I was able to test this behavior on the
almost-10-years-old Ruby 1.8.5, so you should be fine with any version.</p>

<p>You can stop here and try to solve this problem or skip below for an answer.
I’m including an unrelated image below so that people with large screen are not
spoiled.</p>

<p><a href="https://www.flickr.com/photos/bfontaine/16880971668/in/dateposted-public/"><img src="https://farm8.staticflickr.com/7590/16880971668_f5eedc26f4_o.jpg%22" alt="" /></a>
The answer to the problem lies in two things: <em>string literals</em> and <em>string
formatting</em>.</p>

<p>You might know you can use <code>%q()</code> to create a string; which can be handy if you
have both single and double quotes and don’t want to escape them:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='ruby'><span class="n">my_str</span> <span class="o">=</span> <span class="sx">%q(it&#39;s a &quot;valid&quot; string)</span>
</code></pre></div></figure></notextile></div>


<p>This method doesn’t support interpolation with <code>#{}</code> but its uppercased friend
does:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='ruby'><span class="n">my_str</span> <span class="o">=</span> <span class="sx">%Q(i&#39;s still </span><span class="si">#{</span><span class="mi">42</span> <span class="o">-</span> <span class="mi">41</span><span class="si">}</span><span class="sx"> &quot;valid&quot; string)</span>
</code></pre></div></figure></notextile></div>


<p>The <a href="http://ruby-doc.org/core-2.3.0/doc/syntax/literals_rdoc.html#label-Percent+Strings">equivalent</a> also exists to create arrays of strings with <code>%w()</code> and
<code>%W()</code>, regular expressions with <code>%r</code>, as well as <code>%i</code> to create arrays of
symbols starting in Ruby 2.0.0:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='ruby'><span class="n">names</span> <span class="o">=</span> <span class="sx">%w(alice bob charlie)</span> <span class="c1"># =&gt; [&quot;alice&quot;, &quot;bob&quot;, &quot;charlie&quot;]</span>
<span class="n">names</span><span class="o">.</span><span class="n">each</span> <span class="p">{</span> <span class="o">|</span><span class="nb">name</span><span class="o">|</span> <span class="nb">puts</span> <span class="s2">&quot;Hello </span><span class="si">#{</span><span class="nb">name</span><span class="si">}</span><span class="s2">&quot;</span> <span class="p">}</span>

<span class="n">my_syms</span> <span class="o">=</span> <span class="o">%</span><span class="n">i</span><span class="p">(</span><span class="n">my</span> <span class="n">symbols</span><span class="p">)</span> <span class="c1"># =&gt; [:my, :symbols]</span>

<span class="nb">puts</span> <span class="s2">&quot;yep&quot;</span> <span class="k">if</span> <span class="sx">%q(my string)</span> <span class="o">=~</span> <span class="sr">%r(my.+regexp)</span>
</code></pre></div></figure></notextile></div>


<p>You can also use <code>[]</code>,<code>{}</code> or <code>&lt;&gt;</code> instead of parentheses:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='ruby'><span class="sx">%w{foo bar}</span>             <span class="c1"># =&gt; [&quot;foo&quot;, &quot;bar&quot;]</span>
<span class="sx">%q[my string goes here]</span> <span class="c1"># =&gt; &quot;my string goes here&quot;</span>
<span class="o">%</span><span class="n">i</span><span class="o">&lt;</span><span class="n">a</span> <span class="n">b</span> <span class="n">c</span><span class="o">&gt;</span>               <span class="c1"># =&gt; [:a, :b, :c]</span>
</code></pre></div></figure></notextile></div>


<p>Ruby lets you use a percent sign alone as an alias of <code>%Q</code>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='ruby'><span class="sx">%(foo bar)</span> <span class="o">==</span> <span class="sx">%&lt;foo bar&gt;</span> <span class="c1"># =&gt; true</span>
<span class="sx">%{foo bar}</span> <span class="o">==</span> <span class="s2">&quot;foo bar&quot;</span>  <span class="c1"># =&gt; true</span>
</code></pre></div></figure></notextile></div>


<p>But wait; there’s more! You can also use most non-alphanumeric characters like
<code>|</code> (<code>%|my string|</code>), <code>^</code> (<code>%w^x y z^</code>), or… <code>%</code>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='ruby'><span class="sx">%w%my array%</span>        <span class="c1"># =&gt; [&quot;my&quot;, &quot;array&quot;]</span>
<span class="sx">%q%my string%</span>       <span class="c1"># =&gt; &quot;my string&quot;</span>
<span class="sx">%%my other string%</span>  <span class="c1"># =&gt; &quot;my other string&quot;</span>
</code></pre></div></figure></notextile></div>


<p>This means that <code>%||</code>, <code>%^^</code> or <code>%%%</code> can be used to denote an empty string
(don’t do that in real programs, please). It answers the problem for the case
n=0: <code>%%%</code> is an empty string; the first percent sign indicates it’s a literal
string, and the following two are respectively the beginning and end
delimiters.</p>

<p>The second part of our answer is <strong>string formatting</strong>.</p>

<p>If you have ever written a Python program you know it supports string
formatting <i>à la</i> <code>sprintf</code> with <code>%</code>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='python'><span class="k">print</span> <span class="s">&quot;this is </span><span class="si">%s</span><span class="s">, I&#39;m </span><span class="si">%d</span><span class="s"> years-old&quot;</span> <span class="o">%</span> <span class="p">(</span><span class="s">&quot;Python&quot;</span><span class="p">,</span> <span class="mi">25</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>


<p>Well, Ruby supports the same method, called <a href="http://ruby-doc.org/core-2.3.0/String.html#method-i-25"><code>String#%</code></a>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='ruby'><span class="nb">puts</span> <span class="s2">&quot;this is %s, I&#39;m %d years-old&quot;</span> <span class="o">%</span> <span class="o">[</span><span class="s2">&quot;Ruby&quot;</span><span class="p">,</span> <span class="mi">21</span><span class="o">]</span>
</code></pre></div></figure></notextile></div>


<p>In both languages you can drop the array/tuple if you have only one argument:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='python'><span class="k">print</span> <span class="s">&quot;I&#39;m </span><span class="si">%s</span><span class="s">&quot;</span> <span class="o">%</span> <span class="s">&quot;Python&quot;</span>
<span class="k">print</span> <span class="s">&quot;I&#39;m </span><span class="si">%d</span><span class="s">&quot;</span> <span class="o">%</span> <span class="mi">25</span>
</code></pre></div></figure></notextile></div>




<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='ruby'><span class="nb">puts</span> <span class="s2">&quot;I&#39;m %s&quot;</span> <span class="o">%</span> <span class="s2">&quot;Ruby&quot;</span>
<span class="nb">puts</span> <span class="s2">&quot;I&#39;m %d&quot;</span> <span class="o">%</span> <span class="mi">21</span>
</code></pre></div></figure></notextile></div>


<p>Both will raise an exception if you have not enough arguments but only Python
will do it if you have too many of them:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='python'><span class="c"># Python</span>
<span class="k">print</span> <span class="s">&quot;I&#39;m </span><span class="si">%s</span><span class="s">&quot;</span> <span class="o">%</span> <span class="p">[</span><span class="s">&quot;Python&quot;</span><span class="p">,</span> <span class="s">&quot;Ruby&quot;</span><span class="p">]</span>
<span class="c"># =&gt; TypeError: not all arguments converted during string formatting</span>
</code></pre></div></figure></notextile></div>




<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='ruby'><span class="c1"># Ruby</span>
<span class="nb">puts</span> <span class="s2">&quot;I&#39;m %s&quot;</span> <span class="o">%</span> <span class="o">[</span><span class="s2">&quot;Ruby&quot;</span><span class="p">,</span> <span class="s2">&quot;Python&quot;</span><span class="o">]</span>
<span class="c1"># prints &quot;I&#39;m Ruby&quot;</span>
</code></pre></div></figure></notextile></div>


<p>This means that while <code>"" % ""</code> is syntaxically valid in both languages, only
Ruby runs it without error, because Python raises an exception telling that the
argument (the string on the right) is not used.</p>

<p>If we combine this knowledge with what we have above with literal strings we
now know we can write the following in Ruby:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='ruby'><span class="sx">%%%</span> <span class="sx">% %%% </span><span class="c1"># equivalent to &quot;&quot; % &quot;&quot;</span>
</code></pre></div></figure></notextile></div>


<p>The last key is that it works without spaces and can be chained:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='ruby'><span class="s2">&quot;&quot;</span>  <span class="o">%</span> <span class="s2">&quot;&quot;</span>  <span class="o">%</span> <span class="s2">&quot;&quot;</span>  <span class="c1"># =&gt; &quot;&quot;</span>
<span class="sx">%%%</span> <span class="sx">% %%% </span><span class="o">%</span> <span class="sx">%%%</span> <span class="c1"># =&gt; &quot;&quot;</span>
<span class="sx">%%%%%%%%%</span><span class="o">%</span><span class="sx">%  </span>   <span class="c1"># =&gt; &quot;&quot;</span>
</code></pre></div></figure></notextile></div>


<p>The 3+4n refers to the way the expression is constructed: the first three
percent signs are an empty string, and the next four ones are the formatting
operator followed by another empty string.</p>

<hr />

<p>Want more of these? Here are a few other valid Ruby expressions using strings
and percent signs (one per line); guess how they’re parsed and evaluated:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='text'>%*%%*%%*%%*
%_/\|/\_/\|/\__
%_.-&#39;&#39;-._%%_.-&#39;&#39;-._%%_.-&#39;&#39;-_%%.etc.
%/\/\/\/\/\______/
%# &lt;- is it really valid? :-#
</code></pre></div></figure></notextile></div>


feed_id: http://bfontaine.net/blog/2016/11/14/where-to-buy-cheap-shirts-online
title: Where to Buy (Cheap) Shirts Online
uri: Where-to-Buy-(Cheap)-Shirts-Online
url: http://bfontaine.net/blog/2016/11/14/where-to-buy-cheap-shirts-online/
content:
<p>This post title sounds like a spam email subject but I was asked twice in the
past three weeks where I buy my t-shirts, so here’s an answer that everybody
can benefit from.</p>

<p>Living in Europe means we can get quite high shipping costs on US-based
websites. The highest shipping cost per t-shirt I ever got was $23 for two
t-shirts and on some websites up to 40% of the total cost is due to shipping.</p>

<p>The main criteria I have when buying t-shirts online are:</p>

<ol>
<li>Their price. I very rarely buy t-shirts that are more expensive than $20-25.
The reason is I like to buy multiple t-shirts at once and having to pay $100
for four t-shirts plus shipping isn’t something I’d consider.</li>
<li>The quality of their design. Of course if all you want is a unicolor t-shirt
you can go to virtually any clothing store in the street to get one.</li>
<li>Their shipping costs and how much time I’ll have to wait (spoiler: at the
very least a couple weeks if it comes from the US).</li>
</ol>


<p>A bonus point is if they accept Paypal. This is because Paypal does the
Euro/Dollar conversion for me; otherwise if I pay with my credit card in
Dollars my bank <s>steals</s> takes me ~1% of the price, or €1 if it’s lower
than €100.</p>

<p>Not-that-fun story: I once had to pay a $0.02 AWS bill. It cost me €1.02; €1
for my bank and ~€0.02 for AWS.</p>

<p>Note that I’m not affiliated with any of the websites listed in this post and I
bought one or more times on each one.</p>

<h2>Threadless</h2>

<p><a href="https://www.threadless.com">Threadless</a> is the first website I ordered shirts
on; back in 2009. They have a large choice of t-shirts; you have to really dig
to find geeky ones. A lot of them are as low as $10-12 and they have promos
where shirts can go as low as $5. All the ones I bought in 2009 are still in
perfect condition. The shipping costs are acceptable at ~30% of the total cost
(e.g. 5 t-shirts for $45 + $27 for shipping = $72) but I need at least a month
to receive my package.</p>

<p><strong>Pluses</strong>: Large choice, good prices.<br/>
<strong>Minuses</strong>: YMMV but I need to dig a lot to find t-shirts I like.</p>

<h2>6 Dollar Shirts</h2>

<p>I don’t remember how I found <a href="http://6dollarshirts.com/">6 Dollar Shirts</a> but
it was quite a pleasant surprise to see a website that sells shirts at $6 each.
They even have a special price at $50 for ten of them. They have a lot more
geeky shirts and ones with references to popular culture. There are also a lot
of political t-shirts that I’m not interested in — but again you may like them.</p>

<p>The downside of the site are the high shipping costs: $35 for ten shirts. But
given their shirt prices it stays very low: $50+$35 for ten t-shirts means only
$8.5 per t-shirt. The first ones I bought in 2013 are still in pretty good
shape. They’re not as good as Threadless’ but they’re really good for their
price.</p>

<p><strong>Pluses</strong>: Prices, and some designs are really great.<br/>
<strong>Minuses</strong>: After ~20 t-shirts I find it hard to find enough nice shirts to
fill another 10-shirts bulk.</p>

<h2>Teespring</h2>

<p><a href="https://teespring.com">Teespring</a> is a time-limited t-shirts website. People
submit their design and if there are enough buyers before a few days it goes to
print. It’s not a site I often go on but it seems to be popular for when people
want to print a special-occasion shirt.</p>

<p>I bought two shirts from them in 2014; each one cost $18 plus $12 of shipping
cost. That’s quite high, which is why I don’t often visit their website.</p>

<p><strong>Pluses</strong>: Some shirts are nice.<br/>
<strong>Minuses</strong>: Prices are high and there’s not a lot of choice.</p>

<h2>Cotton Bureau</h2>

<p><a href="https://cottonbureau.com/shop">Cotton Bureau</a> is similar to Teespring but with
higher prices and goals appears to be higher than Teespring so less t-shirts go
to print. They do have nice shirts, thought.</p>

<p>Last time I bought a t-shirt from them was on October 31 of this year. I picked
a Mystery Tee as well, which is a cheaper shirt they choose for you. They
should ship on November 28 so I should get them for Christmas — That’s a long
time to wait. I paid $25 for the t-shirt I chose; $15 for the Mystery one; and
$24 for the shipping. You have to really want that t-shirt to pay that much.</p>

<p><strong>Pluses</strong>: Nice designs and if enough people subscribe to an already-printed
t-shirt it goes to print again.<br/>
<strong>Minuses</strong>: Quite expensive and you have to wait until the end of the buying
window to have it printed and shipped.</p>

<h2>Redbubble</h2>

<p><a href="http://www.redbubble.com/">Redbubble</a> is a marketplace that sells a lot of
stuff with designs from a lot of people.</p>

<p>They have a lot of choice but in my experience the quality of their designs is
lower than Threadless’ and 6 Dollar Shirts’. They even sometimes have
duplicated or copied designs: two or three people got the same idea and
submitted (almost) the same design.</p>

<p>The positive side of this is you’ll often find t-shirts that refer to current
TV series that don’t have official stores (unlike <a href="https://secure.cbsstore.com/">CBS</a>’ series), for example <a href="http://www.redbubble.com/people/itsjane/works/15723610-my-other-bike-is-a-tree">this Bojack-Horseman-themed shirt</a>.</p>

<p>Their shipping costs are pretty low; I paid only €2.5 to get my €21 t-shirt.</p>

<p><strong>Pluses</strong>: Low shipping costs and some nice finds if you search a lot.<br/>
<strong>Minuses</strong>: A lot of not-so-original designs that make it harder to find the
good ones.</p>

<h2>TeeFury</h2>

<p><a href="http://www.teefury.com/">TeeFury</a> is a daily t-shirt website. They have two
shirts featured for a day at $11; otherwise you have to pay the full price
($20). I’m not a huge fan of their designs but they have very low shipping
costs; only $3 for a $11 t-shirt.</p>

<p><strong>Pluses</strong>: They’re the cheapest option of this list when you buy only one or
two t-shirts at once.<br/>
<strong>Minuses</strong>: Only two t-shirts per day.</p>

<hr />

<p>I hope this post may help you buy (cheap) t-shirts online. Note the shipping
costs I gave were for me in France; they may vary depending on your location.</p>

<p>Please share your own recommendations in the comments!</p>


feed_id: http://bfontaine.net/blog/2016/09/17/do-you-speak-tar
title: Do You Speak Tar?
uri: Do-You-Speak-Tar
url: http://bfontaine.net/blog/2016/09/17/do-you-speak-tar/
content:
<p><img class="center" src="media/abe90f26.tar.png" data-original-source="http://bfontaine.net/blog/images/tar.png" alt="xkcd comic #1168" ></p>

<p>For a lot of people, the GNU <code>tar</code> command’s options seem obscure and hard to
use. The most common ones exist only in a short form and always appear grouped
in the same order and often without a leading hyphen, e.g. <code>tar xzvf
archive.tgz</code> and not <code>tar -v -z -x -f archive.tgz</code>. Additionally, <code>tar</code> doesn’t
work without any “option”.</p>

<p>These options, or rather these commands, can be seen as a (small) language that
you can learn to <del>speak,</del> write or read. Each command has its own
meaning that sometimes depend on which other commands are used with it.</p>

<h2>The Grammar</h2>

<p><code>tar</code>’s sentences start with a verb. There’s no subject, because you’re giving
an order to <code>tar</code>. This verb is followed by zero or more modifiers that give
more context to the action. The last part is the object(s) on which the action
is made. Spaces are not needed between <code>tar</code>’s words because they all consist
of one letter.</p>

<h3>Actions</h3>

<p>The two most common actions are “<u>c</u>reate” (<code>c</code>) and “e<u>x</u>tract”
(<code>x</code>). The first one is used to <em>create</em> an archive from some files; and the
second is used to <em>extract</em> that archive in order to get back those files.</p>

<p>All <code>tar</code> implementations support one more action: “lis<u>t</u>” (<code>t</code>) to
<em>list</em> an archive’s content without extracting it. Some implementations support
two variants of “<u>c</u>reate” that are “append” (<code>r</code>) and “<u>u</u>pdate”
(<code>u</code>). The former <em>appends</em> files to an existing archive; the latter <em>updates</em>
files in the archive for which there exist a more recent version.</p>

<p>Unfortunately we now know all <code>tar</code> actions but can’t do much without knowing
how to apply them to an object. Let’s dive into objects and we’ll see the
modifiers later.</p>

<h3>Objects</h3>

<p><code>tar</code> has a very limited set of objects: archives. Each <code>tar</code> command operates
on one archive, that is given by <code>f</code> (for “<u>f</u>ile”) followed by its path.</p>

<p>Files added or extracted from archives are simply given as extra arguments to
these commands without needing any special word.</p>

<p>We’re now ready to write our first meaningful sentences.</p>

<p>“Hey <code>tar</code>, please <u>c</u>reate an archive <u>f</u>ile <code>foo.tar</code> with <code>file1</code>
and <code>file2</code>” is written as <code>tar cf foo.tar file1 file2</code>.</p>

<p>“E<u>x</u>tract archive <u>f</u>ile <code>foo.tar</code>” is written as <code>tar xf foo.tar</code>.
“Lis<u>t</u> archive file <code>foo.tar</code>” is written as <code>tar tf foo.tar</code>. You get
the idea.</p>

<p>Note that actions like “e<u>x</u>tract” or “lis<u>t</u>” accept additional
arguments for the file patterns you want to extract/list. Say you have a big
archive from which you only want to extract one <code>important.txt</code> file. Just give
this information to <code>tar</code> and it’ll kindly e<u>x</u>tract it for you:</p>

<pre><code>tar xf big-archive.tar important.txt
</code></pre>

<p>You might wonder what is this “<u>f</u>ile” word for if we <em>always</em> need it.
Well, we can remove it. But if we do so, our <code>tar</code> command doesn’t have any
object left, so it’ll look at something else: <code>STDIN</code> or <code>STDOUT</code>.</p>

<p>Actions that read archives operate on <code>STDIN</code> if you don’t give them a
<u>f</u>ile object:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='sh'>cat big-archive.tar <span class="p">|</span> tar x important.txt
</code></pre></div></figure></notextile></div>


<p>You can also be explicit by giving <code>-</code> to <code>f</code>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='sh'>cat big-archive.tar <span class="p">|</span> tar xf - important.txt
</code></pre></div></figure></notextile></div>


<p>The “<u>c</u>reate” action will output the archive on <code>STDOUT</code> if you don’t
give it a name (or use <code>f -</code>). You still need to give it the name of the files
to put in that archive:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='sh'>tar c file1 file2 &gt; archive.tar
<span class="c"># Same, but more explicit</span>
tar cf - file1 file2 &gt; archive.tar
</code></pre></div></figure></notextile></div>


<p>Note that you can’t e<u>x</u>tract an archive to <code>STDOUT</code> without a modifier.
<code>tar</code> operates on files, not on data streams. By default it doesn’t compress
its content so <u>c</u>reating a <code>tar</code> archive for one file doesn’t make much
sense.</p>

<p>Now that we know how to write basic sentences, let’s add some modifiers to
them.</p>

<h3>Modifiers</h3>

<p>In my experience the most used modifiers are <code>v</code> and <code>z</code>. The first one is the
“<u>v</u>erbose” flag and makes <code>tar</code> more chatty. When <u>c</u>reating or
e<u>x</u>tracting an archive it’ll print each file’s name as it’s (un)archiving
it. When lis<u>t</u>ing an archive it’ll print more info about each file.
Compare both outputs below:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='console'><span class="gp">$</span> tar tf archive.tar
<span class="go">file1</span>
<span class="go">file2</span>
</code></pre></div></figure></notextile></div>




<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='console'><span class="gp">$</span> tar tvf archive.tar
<span class="go">-rw-r--r--  0 baptiste wheel   31425 18 sep 14:51 file1</span>
<span class="go">-rw-r--r--  0 baptiste wheel   18410 18 sep 14:51 file2</span>
</code></pre></div></figure></notextile></div>


<p>The <code>v</code> modifier can be combined with any other one mentioned below.</p>

<p><code>z</code> will tell <code>tar</code> to use G<u>z</u>ip to (de)compress the archive. Nowadays
<code>tar</code>-ing with no compression is rarely used, and Gzip is ubiquitous. Just add
<code>z</code> to your modifiers and <code>tar</code> will <u>c</u>reate compressed archive and
e<u>x</u>tract them. The convention is to use <code>.tar.gz</code> or <code>.tgz</code> for such
archives:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='sh'>tar czf archive.tar.gz file1 file2
<span class="c"># Later…</span>
tar xzf archive.tar.gz
</code></pre></div></figure></notextile></div>


<p>Other common modifiers include <code>j</code> that works exactly like <code>z</code> but (de)compress
using Bzip2 instead of Gzip. Such archives usually end with <code>.tar.bz2</code> or
<code>.tbz2</code>. It’s not named <code>b</code> or <code>B</code> because <a href="https://superuser.com/q/441563/95929#comment511048_441563">those were already taken</a> when
this modifier was introduced.</p>

<p>Similarly to <code>j</code> one can use its capital friend, <code>J</code>. This one compresses using
<code>xz</code> instead of the last Gzip or Bzip2. These archives use the extensions
<code>.tar.xz</code> or <code>.txz</code>.</p>

<p>Note you can also (de)compress archives by yourself if you don’t remember these
modifiers:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='sh'>tar cf myarchive.tar file1 file2
gzip myarchive.tar
<span class="c"># Later…</span>
gunzip myarchive.tar
tar xf myarchive.tar
</code></pre></div></figure></notextile></div>


<p>There is a dozen of other modifiers you can find in the manpage, but let’s
mention two more: <code>O</code> and <code>k</code>. You may remember from the first section that I
wrote you can’t e<u>x</u>tract an archive to <code>STDOUT</code> without a modifier. Well,
that modifier is called <code>O</code>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='sh'>tar xOf myarchive.tar
</code></pre></div></figure></notextile></div>


<p>Using <code>O</code> when e<u>x</u>tracting will print the content of the archive on
<code>STDOUT</code>. This is the same output as you would get by calling <code>cat</code> on all the
files in it.</p>

<p>The last modifier I wanted to mention is <code>k</code>, which tells <code>tar</code> not to override
existing files when e<u>x</u>tracting an archive. That is, if you already have
a file called <code>important.txt</code> in your directory and you un-<code>tar</code> an archive
using the <code>k</code> modifier, you can be sure it won’t override your existing
<code>important.txt</code> file.</p>

<hr />

<p>I hope this post helped you have a better understanding of <code>tar</code> commands, and
how they’re not <em>that</em> complicated. I put a few (valid) commands below, just so
you can see if you understand what’s they’re doing:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='sh'>tar xf foo.tar
tar cvzf bar.tar a b c
tar tvf foo.tar important.txt
tar cz file1 file2 &gt; somefile
tar xzOf somefile
tar cJf hey.txz you andyou
tar xkjvf onemore.tbz2
</code></pre></div></figure></notextile></div>


feed_id: https://bfontaine.net/blog/2017/03/04/add-this-directory-to-your-path
title: Add This Directory to Your PATH
uri: Add-This-Directory-to-Your-PATH
url: https://bfontaine.net/blog/2017/03/04/add-this-directory-to-your-path/
content:
<p>You’ve probably read this instruction quite a few times in online tutorials
about installing command-line tools. What is this <code>PATH</code> for? Why should we
add directories &ldquo;to&rdquo; it? This is the subject of this post.</p>

<h2>How the Shell Finds Executables</h2>

<p>When you type something like <code>ls</code>, your shell has to find what is this <code>ls</code>
program you’re trying to run. Typical shells only have a handful predefined
commands like <code>cd</code>; most of the commands you use everyday are standalone
programs.</p>

<p>In order to find that <code>ls</code> program; your shell looks in a few directories.
Those directories are stored in an environment variable called <code>PATH</code>; you can
look up its value in most shells using the following command:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='sh'><span></span><span class="nb">echo</span> <span class="nv">$PATH</span>
</code></pre></div></figure></notextile></div>


<p>You can see it contains a list of paths separated by colons. Finding a program
in these is just a matter of checking each directory to see if it contains an
executable with the given name. This lookup is implemented by the <code>which</code>
program. The algorithm is pretty simple and goes like this:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='text'><span></span>cmd = &quot;ls&quot;
for directory in $PATH.split(&quot;:&quot;) {
    candidate = $directory/$cmd ;
    if candidate exists and is executable {
        return candidate ;
    }
}
</code></pre></div></figure></notextile></div>


<p>This is pseudo-code, but there are implementations in various languages:
<a href="https://github.com/clibs/which/blob/641a91a8a509877e1f27d223299e6615472c9ec8/src/which.c#L35-L36">C</a>, <a href="https://github.com/bfontaine/which/blob/eb85e63e18a74984175ed2425b148f06a0428431/which/which.go#L41-L57">Go</a>, <a href="https://github.com/Homebrew/brew/blob/370c711da23fa0f7c7944f7d15928e44a67656a9/Library/Homebrew/utils.rb#L299-L311">Ruby</a>, <a href="https://github.com/dart-lang/which/blob/b92330b876b832e3c632629288d0ee1eca9571c5/lib/src/which_impl.dart#L38">Dart</a>, <a href="https://github.com/search?q=which">etc</a>.</p>

<p>As you can see, the order of these directories matters because the first
matching candidate is used. That means if you have a custom <code>ls</code> program in
<code>/my/path</code>, putting <code>/my/path</code> at the beginning of the <code>PATH</code> variable will
cause <code>ls</code> to always refer to <em>your</em> version instead of e.g. <code>/bin/ls</code> because
<code>/bin</code> appears <em>after</em> <code>/my/path</code>.</p>

<p>You can perform this lookup using the <code>which</code> command. Add <code>-a</code> to get all
matching executables in the order in which they’re found. This is what
<code>which python</code> gives on my machine:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='text'><span></span>$ which python
/usr/local/bin/python

$ which -a python
/usr/local/bin/python
/usr/bin/python
</code></pre></div></figure></notextile></div>


<p>You can see I have two <code>python</code>’s but the shell picks the one in
<code>/usr/local/bin</code> instead of <code>/usr/bin</code> because it appears first in my <code>PATH</code>.
You can bypass this lookup by giving a <em>path</em> to your executable. This is why
you run executables in the current directory by prefixing them with <code>./</code>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='text'><span></span>./my-program
</code></pre></div></figure></notextile></div>


<p>This tells the shell you want to run the program called <code>my-program</code> in the
<em>current</em> directory. It won’t search in the <code>PATH</code> for it. It also works with
absolute paths. The following command runs my <code>python</code> in <code>/usr/bin</code> regardless
of what’s in my <code>PATH</code> variable:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='text'><span></span>/usr/bin/python
</code></pre></div></figure></notextile></div>


<p>For performance reasons a shell like Bash won’t look executables up all the
time. It’ll cache the information for the current session and will hence do
this lookup only once per command. This is why you must reload your shell to
have your <code>PATH</code> modifications taken into account. You can force Bash to clear
its current cache with the <code>hash</code> builtin:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span></span><span class="nb">hash</span> -r
</code></pre></div></figure></notextile></div>


<p>Now that we know how our shell find executables; let’s see how this <code>PATH</code>
variable is populated.</p>

<h2>Where Does That <code>PATH</code> Come From?</h2>

<p>This part depends on both your shell and your operating system. Bash reads
<code>/etc/profile</code> when it starts. It contains some setup instructions, including
initial values for the <code>PATH</code> variable. On macOS, it executes
<code>/usr/libexec/path_helper</code> which in turns looks in <code>/etc/paths</code> for the initial
paths.</p>

<p>The file looks like this on my machine:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='text'><span></span>$ cat /etc/paths
/usr/bin
/bin
/usr/sbin
/sbin
</code></pre></div></figure></notextile></div>


<p>The actual code to set the <code>PATH</code> variable (or any variable for that matter) in
Bash is below:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span></span><span class="nv">PATH</span><span class="o">=</span><span class="s2">&quot;/one/directory:/another/directory:/another/one&quot;</span>
</code></pre></div></figure></notextile></div>


<p>By default, Bash doesn’t pass its variables to the child processes. That is, if
you set a variable in Bash then try to use it in a program it’ll fail:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='text'><span></span>$ myvar=foo
$ ruby -e &#39;puts ENV[&quot;myvar&quot;] || &quot;nope :(&quot;&#39;
nope :(
</code></pre></div></figure></notextile></div>


<p>Bash allows one to mark a variable as exported to subprocesses with the <code>export</code> builtin command:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='text'><span></span>$ myvar=foo
$ export myvar
$ ruby -e &#39;puts ENV[&quot;myvar&quot;] || &quot;nope :(&quot;&#39;
foo
</code></pre></div></figure></notextile></div>


<p>This is usually done when setting the variable:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='text'><span></span>$ export myvar=foo
$ ruby -e &#39;puts ENV[&quot;myvar&quot;] || &quot;nope :(&quot;&#39;
foo
</code></pre></div></figure></notextile></div>


<p>Technically Bash doesn’t need you to export the <code>PATH</code> variable to use it but
it’s better if for example a program you use executes another program; in this
case the former must be able to find the latter using the correct <code>PATH</code>.</p>

<h2>How Do We Modify It?</h2>

<p>Each shell has its own file in the user directory to allow per-user setup
scripts. For Bash, it’s <code>~/.bash_profile</code>, which often sources <code>~/.bashrc</code>.
You can use this file to override the default <code>PATH</code>. It’ll be loaded when
starting a session; meaning you have to either reload your shell either
re-source this file after modifying it.</p>

<p>We saw in the previous section how to set the <code>PATH</code> variable; but most of the
time we don’t want to manually set the whole directories list; we only want to
<em>modify</em> its value to add our own directories.</p>

<p>We won’t dive into the details in that post, but Bash has a syntax to get the
value of a variable by prefixing it with a dollar symbol:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span></span><span class="nb">echo</span> myvar  <span class="c1"># prints &quot;myvar&quot;</span>
<span class="nb">echo</span> <span class="nv">$myvar</span> <span class="c1"># prints &quot;foo&quot;, i.e. myvar&#39;s value</span>
</code></pre></div></figure></notextile></div>


<p>Bash also supports string interpolation using double quotes: You can include
the value of a variable in a double-quotes string by just writing <code>$</code> followed
by its name:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span></span><span class="nb">echo</span> <span class="s2">&quot;hello I&#39;m </span><span class="nv">$myvar</span><span class="s2">&quot;</span>  <span class="c1"># prints &quot;hello I&#39;m foo&quot;</span>
</code></pre></div></figure></notextile></div>


<p>We use this feature to append or prepend directories to the <code>PATH</code> variable:
prepending means setting the <code>PATH</code>’s value to that directory followed by a
colon followed by the previous <code>PATH</code>’s value:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span></span><span class="nv">PATH</span><span class="o">=</span><span class="s2">&quot;/my/directory:</span><span class="nv">$PATH</span><span class="s2">&quot;</span>
</code></pre></div></figure></notextile></div>


<p>You usually don’t need to re-mark this variable as exported but using <code>export</code>
at the beginning of the command doesn’t hurt.</p>

<h2>Wrapping Things Up</h2>

<p>Modifying the <code>PATH</code> is not something we do very often because most tools are
installed in standard locations—already in our <code>PATH</code>. Most package managers
install executables in their own location and need the user to modify their
<code>PATH</code>. <a href="https://brew.sh/">Homebrew</a>, for example, installs them under <code>/usr/local/bin</code> and
<code>/usr/local/sbin</code> by default. If those are not already in the <code>PATH</code>, one needs
to add them:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span></span><span class="c1"># In e.g. ~/.bash_profile</span>
<span class="nb">export</span> <span class="nv">PATH</span><span class="o">=</span><span class="s2">&quot;/usr/local/bin:/usr/local/sbin:</span><span class="nv">$PATH</span><span class="s2">&quot;</span>
</code></pre></div></figure></notextile></div>


<p>This means the shell will first look in these directories for executables. It
allows one to “override” existing tools with more up-to-date ones.</p>


feed_id: https://bfontaine.net/blog/2016/09/17/do-you-speak-tar
title: Do You Speak Tar?
uri: 2017-03-05-Do-You-Speak-Tar
url: https://bfontaine.net/blog/2016/09/17/do-you-speak-tar/
content:
<p><img class="center" src="media/105b6ed5.tar.png" data-original-source="https://bfontaine.net/blog/images/tar.png" alt="xkcd comic #1168" ></p>

<p>For a lot of people, the GNU <code>tar</code> command’s options seem obscure and hard to
use. The most common ones exist only in a short form and always appear grouped
in the same order and often without a leading hyphen, e.g. <code>tar xzvf
archive.tgz</code> and not <code>tar -v -z -x -f archive.tgz</code>. Additionally, <code>tar</code> doesn’t
work without any “option”.</p>

<p>These options, or rather these commands, can be seen as a (small) language that
you can learn to <del>speak,</del> write or read. Each command has its own
meaning that sometimes depend on which other commands are used with it.</p>

<h2>The Grammar</h2>

<p><code>tar</code>’s sentences start with a verb. There’s no subject, because you’re giving
an order to <code>tar</code>. This verb is followed by zero or more modifiers that give
more context to the action. The last part is the object(s) on which the action
is made. Spaces are not needed between <code>tar</code>’s words because they all consist
of one letter.</p>

<h3>Actions</h3>

<p>The two most common actions are “<u>c</u>reate” (<code>c</code>) and “e<u>x</u>tract”
(<code>x</code>). The first one is used to <em>create</em> an archive from some files; and the
second is used to <em>extract</em> that archive in order to get back those files.</p>

<p>All <code>tar</code> implementations support one more action: “lis<u>t</u>” (<code>t</code>) to
<em>list</em> an archive’s content without extracting it. Some implementations support
two variants of “<u>c</u>reate” that are “append” (<code>r</code>) and “<u>u</u>pdate”
(<code>u</code>). The former <em>appends</em> files to an existing archive; the latter <em>updates</em>
files in the archive for which there exist a more recent version.</p>

<p>Unfortunately we now know all <code>tar</code> actions but can’t do much without knowing
how to apply them to an object. Let’s dive into objects and we’ll see the
modifiers later.</p>

<h3>Objects</h3>

<p><code>tar</code> has a very limited set of objects: archives. Each <code>tar</code> command operates
on one archive, that is given by <code>f</code> (for “<u>f</u>ile”) followed by its path.</p>

<p>Files added or extracted from archives are simply given as extra arguments to
these commands without needing any special word.</p>

<p>We’re now ready to write our first meaningful sentences.</p>

<p>“Hey <code>tar</code>, please <u>c</u>reate an archive <u>f</u>ile <code>foo.tar</code> with <code>file1</code>
and <code>file2</code>” is written as <code>tar cf foo.tar file1 file2</code>.</p>

<p>“E<u>x</u>tract archive <u>f</u>ile <code>foo.tar</code>” is written as <code>tar xf foo.tar</code>.
“Lis<u>t</u> archive file <code>foo.tar</code>” is written as <code>tar tf foo.tar</code>. You get
the idea.</p>

<p>Note that actions like “e<u>x</u>tract” or “lis<u>t</u>” accept additional
arguments for the file patterns you want to extract/list. Say you have a big
archive from which you only want to extract one <code>important.txt</code> file. Just give
this information to <code>tar</code> and it’ll kindly e<u>x</u>tract it for you:</p>

<pre><code>tar xf big-archive.tar important.txt
</code></pre>

<p>You might wonder what is this “<u>f</u>ile” word for if we <em>always</em> need it.
Well, we can remove it. But if we do so, our <code>tar</code> command doesn’t have any
object left, so it’ll look at something else: <code>STDIN</code> or <code>STDOUT</code>.</p>

<p>Actions that read archives operate on <code>STDIN</code> if you don’t give them a
<u>f</u>ile object:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='sh'>cat big-archive.tar <span class="p">|</span> tar x important.txt
</code></pre></div></figure></notextile></div>


<p>You can also be explicit by giving <code>-</code> to <code>f</code>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='sh'>cat big-archive.tar <span class="p">|</span> tar xf - important.txt
</code></pre></div></figure></notextile></div>


<p>The “<u>c</u>reate” action will output the archive on <code>STDOUT</code> if you don’t
give it a name (or use <code>f -</code>). You still need to give it the name of the files
to put in that archive:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='sh'>tar c file1 file2 &gt; archive.tar
<span class="c"># Same, but more explicit</span>
tar cf - file1 file2 &gt; archive.tar
</code></pre></div></figure></notextile></div>


<p>Note that you can’t e<u>x</u>tract an archive to <code>STDOUT</code> without a modifier.
<code>tar</code> operates on files, not on data streams. By default it doesn’t compress
its content so <u>c</u>reating a <code>tar</code> archive for one file doesn’t make much
sense.</p>

<p>Now that we know how to write basic sentences, let’s add some modifiers to
them.</p>

<h3>Modifiers</h3>

<p>In my experience the most used modifiers are <code>v</code> and <code>z</code>. The first one is the
“<u>v</u>erbose” flag and makes <code>tar</code> more chatty. When <u>c</u>reating or
e<u>x</u>tracting an archive it’ll print each file’s name as it’s (un)archiving
it. When lis<u>t</u>ing an archive it’ll print more info about each file.
Compare both outputs below:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='console'><span class="gp">$</span> tar tf archive.tar
<span class="go">file1</span>
<span class="go">file2</span>
</code></pre></div></figure></notextile></div>




<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='console'><span class="gp">$</span> tar tvf archive.tar
<span class="go">-rw-r--r--  0 baptiste wheel   31425 18 sep 14:51 file1</span>
<span class="go">-rw-r--r--  0 baptiste wheel   18410 18 sep 14:51 file2</span>
</code></pre></div></figure></notextile></div>


<p>The <code>v</code> modifier can be combined with any other one mentioned below.</p>

<p><code>z</code> will tell <code>tar</code> to use G<u>z</u>ip to (de)compress the archive. Nowadays
<code>tar</code>-ing with no compression is rarely used, and Gzip is ubiquitous. Just add
<code>z</code> to your modifiers and <code>tar</code> will <u>c</u>reate compressed archive and
e<u>x</u>tract them. The convention is to use <code>.tar.gz</code> or <code>.tgz</code> for such
archives:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='sh'>tar czf archive.tar.gz file1 file2
<span class="c"># Later…</span>
tar xzf archive.tar.gz
</code></pre></div></figure></notextile></div>


<p>Other common modifiers include <code>j</code> that works exactly like <code>z</code> but (de)compress
using Bzip2 instead of Gzip. Such archives usually end with <code>.tar.bz2</code> or
<code>.tbz2</code>. It’s not named <code>b</code> or <code>B</code> because <a href="https://superuser.com/q/441563/95929#comment511048_441563">those were already taken</a> when
this modifier was introduced.</p>

<p>Similarly to <code>j</code> one can use its capital friend, <code>J</code>. This one compresses using
<code>xz</code> instead of the last Gzip or Bzip2. These archives use the extensions
<code>.tar.xz</code> or <code>.txz</code>.</p>

<p>Note you can also (de)compress archives by yourself if you don’t remember these
modifiers:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='sh'>tar cf myarchive.tar file1 file2
gzip myarchive.tar
<span class="c"># Later…</span>
gunzip myarchive.tar
tar xf myarchive.tar
</code></pre></div></figure></notextile></div>


<p>There is a dozen of other modifiers you can find in the manpage, but let’s
mention two more: <code>O</code> and <code>k</code>. You may remember from the first section that I
wrote you can’t e<u>x</u>tract an archive to <code>STDOUT</code> without a modifier. Well,
that modifier is called <code>O</code>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='sh'>tar xOf myarchive.tar
</code></pre></div></figure></notextile></div>


<p>Using <code>O</code> when e<u>x</u>tracting will print the content of the archive on
<code>STDOUT</code>. This is the same output as you would get by calling <code>cat</code> on all the
files in it.</p>

<p>The last modifier I wanted to mention is <code>k</code>, which tells <code>tar</code> not to override
existing files when e<u>x</u>tracting an archive. That is, if you already have
a file called <code>important.txt</code> in your directory and you un-<code>tar</code> an archive
using the <code>k</code> modifier, you can be sure it won’t override your existing
<code>important.txt</code> file.</p>

<hr />

<p>I hope this post helped you have a better understanding of <code>tar</code> commands, and
how they’re not <em>that</em> complicated. I put a few (valid) commands below, just so
you can see if you understand what’s they’re doing:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='sh'>tar xf foo.tar
tar cvzf bar.tar a b c
tar tvf foo.tar important.txt
tar cz file1 file2 &gt; somefile
tar xzOf somefile
tar cJf hey.txz you andyou
tar xkjvf onemore.tbz2
</code></pre></div></figure></notextile></div>


feed_id: https://bfontaine.net/blog/2014/11/11/a-python-toolbox
title: A Python toolbox
uri: 2017-03-05-A-Python-toolbox
url: https://bfontaine.net/blog/2014/11/11/a-python-toolbox/
content:
<p>I started learning Python four years ago and have been heavily programming with
it for near than a year now. In this post I’ll share some tools I use to ease
and speed-up my workflow, either in the Python code or in the development
environment.</p>

<p>These tips should be OS- and editor-independent. I have some useful Vim plugins
to work with Python, but that’ll be for another post. You might have to adapt
commands if you work on Windows.</p>

<h2>Setting Things Up</h2>

<p>Let’s say you’re starting a Python library. You have a couple dependencies, and
hopefully you’d like it to work on multiple versions, like 2.6, 2.7, and 3.x.
How can you test that? You have to (a) manage your dependencies to not mess up
with your user environment, and (b) test with multiple Python versions.</p>

<p>Introducing <a href="http://virtualenv.readthedocs.org/en/latest/virtualenv.html">virtualenv</a>.</p>

<p>Virtualenv lets you create <em>isolated</em> Python environments. That means you get a
pristine environment where you’ll install your library’s dependencies, and
nothing else. It’ll be independent of your user space. It’s important to work
with an isolated environment because you don’t know which environment will your
users have, so you shouldn’t make any assumption besides your own requirements.
Working with your user environment means you might forgot a dependency because
it just works since it’s installed on your computer but it’ll broke if run on
another computer without this dependency.</p>

<p>You should be able to install it with <code>pip</code>:</p>

<pre><code>$ pip install virtualenv
</code></pre>

<p>It needs a directory to store the environment. I usually use <code>venv</code>, but you
can choose whatever you want:</p>

<pre><code>$ virtualenv venv
</code></pre>

<p>You can then either “activate” the environment, which adds its <code>bin</code> directory
in your <code>PATH</code>:</p>

<pre><code>$ source venv/bin/activate
$ python  # that's your virtualenv's Python
$ deactivate
$ python  # that's your Python
</code></pre>

<p>or prefix your commands with <code>venv/bin/</code> (replace <code>venv</code> with your directory’s
name):</p>

<pre><code>$ venv/bin/python  # that's your virtualenv's Python
$ python  # that's your Python
</code></pre>

<p>I usually do the later. Install dependencies in the environment:</p>

<pre><code>$ venv/bin/pip install your-dependency
</code></pre>

<p>Don’t forget to tell Git or any tool you’re using for versioning to ignore
this <code>venv</code> directory. It can take some place (from 12MB to more than 40MB)
depending on the number of dependencies you’re relying on.</p>

<p>To remove an environment, just delete its directory:</p>

<pre><code>$ rm -rf venv
</code></pre>

<p>It can be convenient to save place on your computer if you have dozen of
environments for different projects, especially if can quickly re-create any
environment with its dependencies.</p>

<p>If you’re using <code>pip</code> to manage them, you should know you can install
dependencies not only from the command line, but also from a file, with one
dependency per line. Each one of them is processed as if it were given on the
command-line.</p>

<p>For example, you could have <a href="https://github.com/bfontaine/term2048/blob/master/requirements.txt">a file</a> containing this:</p>

<pre><code>colorama==0.2.7
coverage==3.7.1
pep8==1.4.6
py==1.4.20
tox==1.7.0
argparse&gt;=1.1
virtualenv==1.11.4
</code></pre>

<p>This file is usually called <code>requirements.txt</code>, but here again you can call it
whatever you want. You call install all these at once with this command:</p>

<pre><code>$ pip install -r requirements.txt
</code></pre>

<p>But we’re programmers and we’re lazy, we don’t want to track down each
installed library to include it in this file.</p>

<p>Here comes <code>pip freeze</code>.</p>

<p><code>pip freeze</code> outputs all installed libraries with their version. It can be put
in our <code>requirements.txt</code> for later use:</p>

<pre><code>$ pip freeze &gt; requirements.txt
</code></pre>

<p>This <code>requirements.txt</code> file becomes handy when used with <code>virtualenv</code> because
you’re now able to fire up a new environment and install all required
libraries, with two commands:</p>

<pre><code>$ virtualenv venv
$ venv/bin/pip install -r requirements.txt
</code></pre>

<p>Note that these are libraries used in the environment, not necessarily your
library’s dependencies. In the above example you can see we’re installing
<code>coverage</code> and <code>pep8</code>, which respectively are a code coverage test tool and a
lint one we’ll talk about later in this post, not libraries we’re depending on
here.</p>

<p>You should thus add this file in your public repository, because it provides
anyone with the informations they need to have in order to mirror your
environment and be able to contribute to your project.</p>

<h2>Coding</h2>

<p>Now that your local environment is ready, you can start coding your library.
You’ll often have to fire an interpreter to test some things, use <code>help()</code> to
check a function’s arguments, etc. Having to type the same things over and over
takes time, and remember: we’re lazy.</p>

<p>Like Bash and some other tools, the Python interpreter can be configured with
an user file, namely <code>$PYTHONSTARTUP</code>. It allows you to add autocompletion,
import common modules, and execute pretty much any code you want.</p>

<p>Start by setting <code>PYTHONSTARTUP</code> in your <code>~/.bashrc</code> (if you’re using Bash):</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span class="nb">export </span><span class="nv">PYTHONSTARTUP</span><span class="o">=</span><span class="s2">&quot;$HOME/.pythonrc.py&quot;</span>
</code></pre></div></figure></notextile></div>


<p>Here we’re telling the interpreter to look for the <code>$HOME/.pythonrc.py</code> file
and executing it before giving us a prompt.</p>

<p>Let’s initialize this file with autocompletion support:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='python'><span class="k">try</span><span class="p">:</span>
    <span class="kn">import</span> <span class="nn">readline</span>
<span class="k">except</span> <span class="ne">ImportError</span><span class="p">:</span>
    <span class="k">pass</span>
<span class="k">else</span><span class="p">:</span>
    <span class="kn">import</span> <span class="nn">rlcompleter</span>
    <span class="n">readline</span><span class="o">.</span><span class="n">parse_and_bind</span><span class="p">(</span><span class="s">&quot;tab: complete&quot;</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>


<p>You can add a lot more <a href="http://www.sontek.net/blog/2010/12/28/tips_and_tricks_for_the_python_interpreter.html">stuff</a> in this file, like <a href="https://github.com/bfontaine/Dotfiles/blob/master/.pythonrc.py#L21-L38">history</a>
support, <a href="https://github.com/coderholic/config/blob/master/.pythonrc.py#L77-L80">colored prompts</a> or common imports. For example, if you use <code>sys</code>
and <code>re</code> a lot, you can save time by adding them in your startup file:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='python'><span class="kn">import</span> <span class="nn">sys</span>
<span class="kn">import</span> <span class="nn">re</span>
</code></pre></div></figure></notextile></div>


<p>You won’t need to type these two lines anymore in your interpreter. It doesn’t
change how Python executes files, just the interactive interpreter.</p>

<h2>Testing</h2>

<p>Four different kinds of tests are covered by this part: style checkers to
ensure your code’s consistency, static analysis tools to detect problems before
executing the code, unit tests to actually test your library, and code coverage
tests to ensure you do test <em>all</em> the code.</p>

<h3>Style Checking</h3>

<p>These are tools which help you maintaining a consistent coding style in your
whole codebase. Most of these tools are easy to use, the hardest part being to
choose which one fits your requirements.</p>

<p>Python has a PEP (<q>Python Enhancement Proposal</q>, sort of RFC), the
<a href="http://legacy.python.org/dev/peps/pep-0008/">PEP&nbsp;8</a>, dedicated to its coding conventions. If you want to follow
it, a command-line tool, rightly named <code>pep8</code>, is available.</p>

<pre><code>$ venv/bin/pip install pep8
$ venv/bin/pep8 your/lib/root/directory
...
foo/mod.py:84:80: E501 line too long (96 &gt; 79 characters)
foo/mod.py:85:6: E203 whitespace before ':'
foo/mod.py:86:80: E501 line too long (87 &gt; 79 characters)
foo/mod.py:87:4: E121 continuation line indentation is not a multiple of four
...
</code></pre>

<p>It’ll check each file and print a list of warnings. You can choose to hide some
of them, or use a white list to decide which ones you want. It’s a good tool if
you want to follow the PEP&nbsp;8 conventions.</p>

<p>Another highly customizable tool is <code>pylint</code>. It reads its configuration from
a file in your project, which can <a href="http://stackoverflow.com/a/22449845/735926">inherit</a> from global and user
configurations. It’ll warn you about bad naming, missing docstrings, functions
which take too many arguments, duplicated code, etc. It also gives you some
statistics about your code. It’s really powerful but can be a pain if you don’t
configure it. For example, it warns you about one-letter variables while you
might find them ok.</p>

<p>Enters <code>prospector</code>.</p>

<p>Prospector is built on top of <code>pep8</code> and <code>pylint</code> and comes with sane defaults
regarding the pickiness of both tools. You can <a href="http://blog.landscape.io/prospector-python-static-analysis-for-humans.html">tell it</a> to only
print important problems about your code:</p>

<pre><code>$ venv/bin/pip install prospector
$ venv/bin/prospector --strictness high
</code></pre>

<p>You’ll get a much shorter output, which will hopefully help you find potential
problems in your code.</p>

<h3>Static Analysis</h3>

<p>Here, we’re talking about analysing the code without executing it. Compiled
languages benefit from this at compilation time, but interpreted languages like
Python have no way to have it.</p>

<p>One of the most popular tools for static analysis on Python code is
<a href="https://launchpad.net/pyflakes">Pyflakes</a>. It doesn’t check your coding style like <code>pep8</code> or <code>pylint</code>, but
warns you about missing imports, dead code, unused variables, redefined
functions and more. You can work without style checkers, but static analysis is
really helpful to detect potential bugs before actually running the code.</p>

<p>Pyflakes can be integrated in editors like Vim with <a href="https://github.com/scrooloose/syntastic#readme">Syntastic</a>, but its
command-line usage is as easy as the previous tools:</p>

<pre><code>$ venv/bin/pip install pyflakes
$ venv/bin/pyflakes your/directory
</code></pre>

<p>Prospector, mentioned in the previous section, also includes <code>pyflakes</code>. You
might also want to try <a href="https://pypi.python.org/pypi/flake8">Flake8</a>, which combines <code>pyflakes</code> and <code>pep8</code>.</p>

<h3>Unit Tests</h3>

<p>When talking about <q>testing</q>, we usually think of <em>unit</em> testing, which is
testing small pieces of our code at a time, to make sure everything is working
correctly. The goal is to test only one feature at a time, to quickly be able
to find which parts of the code are not working. There are a lot of great
testing frameworks, and Python comes with a built-in one, <a href="https://docs.python.org/2/library/unittest.html">unittest</a>, which I
personnally use. I won’t cover these, and I’ll instead cover the case when you
need to test on multiple Python versions, which is often the case when you
plan to release a public library. You obviously don’t want to manually switch
to each Python version, install your dependencies then run your tests suit each
time.</p>

<p>This is a job for <code>tox</code>.</p>

<p><a href="https://testrun.org/tox/latest/">Tox</a> uses Virtualenv, which I talked about earlier, to create standalone
Python environments for different Python versions, and test your code in each
one of them.</p>

<pre><code>$ venv/bin/pip install tox
</code></pre>

<p>Like some previous tools, it needs a configuration file. Here is a basic
<code>tox.ini</code> to test on Python 2.6, 2.7, 3.3 and 3.4:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='ini'><span class="k">[tox]</span>
<span class="na">envlist</span> <span class="o">=</span> <span class="s">py26, py27, py33, py34</span>
<span class="na">downloadcache</span> <span class="o">=</span> <span class="s">{toxworkdir}/_download/</span>

<span class="k">[testenv]</span>
<span class="na">sitepackages</span> <span class="o">=</span> <span class="s">False</span>
<span class="na">deps</span> <span class="o">=</span><span class="s"></span>
<span class="s">  colorama</span>
<span class="na">commands</span> <span class="o">=</span><span class="s"></span>
<span class="s">  {envpython} {toxinidir}/tests/test.py</span>
</code></pre></div></figure></notextile></div>


<p>It declares one dependency, <code>colorama</code>, and tells <code>tox</code> to run tests by
executing <code>tests/test.py</code>. That’s all. We can then run our tests:</p>

<pre><code>$ venv/bin/pip/tox
</code></pre>

<p>It’ll takes some time on the first run to fetch dependencies and create
environments, but all the following times will be faster.</p>

<p>Like <code>virtualenv</code>, <code>tox</code> uses a directory to store these environments. You can
safely delete it if you need more space, it’ll be re-created by <code>tox</code> the next
time:</p>

<pre><code>$ rm -rf .tox
</code></pre>

<h3>Code Coverage</h3>

<p>This last part about testing talks about code coverage tests. These are tests
about tests. The goal here is to ensure your tests cover all your code, and
you don’t leave some parts untested. Most tools tell you how many lines where
executed when running your tests suit, and give you an overall coverage
percentage.</p>

<p>One of them is <code>coverage</code>.</p>

<pre><code>$ venv/bin/pip install coverage
</code></pre>

<p>Give it your project’s root directory as well as a file to run your tests:</p>

<pre><code>$ venv/bin/coverage run --source=your/directory tests/test.py
</code></pre>

<p>It’ll run them, and give you a nice coverage report:</p>

<pre><code>$ coverage report -m
Name              Stmts   Miss  Cover   Missing
-----------------------------------------------
foobar/__init__       5      0   100%
foobar/base          46      2    96%   5-6
foobar/cli          159    159     0%   3-280
foobar/config        66     66     0%   3-150
foobar/session       64     14    78%   9-10, 106-124
foobar/barfooqux     22      0   100%
foobar/helloworld    25     20    20%   14-41
...
-----------------------------------------------
TOTAL               459    313    32%
</code></pre>

<p>It can also give you an HTML version:</p>

<pre><code>$ venv/bin/coverage html
</code></pre>

<p>Getting to 100% is the ultimate goal, but you’ll quickly find that’s the first
80% are easy and the remaining 20% are the hardest part, especially when you
have I/O, external dependencies like databases, and/or complicated
corner-cases.</p>

<p>Check <a href="http://nedbatchelder.com/code/coverage/">Coverage</a>’s doc for more info.</p>

<h2>Debugging</h2>

<p>There are a lot of ways to debug, including logs, but the simplest debugging
tool is the good old <code>print</code>. It becomes really impracticable when you have to
restart your server every time you add or remove one of them from your code.
What if you could fire a Python interpreter right from your code and inspect it
when it’s running? Well, you can do that with Python’s <code>code</code> module! This
trick is really handy, and I’ve been using it heavily instead of these <code>print</code>s
we write everywhere since I’ve discovered it.</p>

<p>The <code>code</code> module provides you with an <code>interact</code> function, which takes a
<code>dict</code> of variables to inject in the interpreter. You’ll be able to print them,
play with them, but these changes won’t be reflected in the program you’re
debugging.</p>

<p>Remember that Python lets you get all global variables as a <code>dict</code> with
<code>globals()</code> and all local ones with <code>locals()</code>. We thus start by creating a
mirror of the local environment:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='python'><span class="nb">vars</span> <span class="o">=</span> <span class="nb">globals</span><span class="p">()</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
<span class="nb">vars</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="nb">locals</span><span class="p">())</span>
</code></pre></div></figure></notextile></div>


<p>These two lines get all global variables (including imported modules) in a
<code>dict</code> called <code>vars</code> and add local variables in it. This can then be passed
directly to the interpreter:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='python'><span class="kn">import</span> <span class="nn">code</span>
<span class="n">code</span><span class="o">.</span><span class="n">interact</span><span class="p">(</span><span class="n">local</span><span class="o">=</span><span class="nb">vars</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>


<p>This will start an interpreter with all these variables already available in
it. There’s nothing to install, this is a <a href="https://docs.python.org/2/library/code.html#code.interact">standard Python module</a>.</p>

<p>You can even inline the code and add it in your favorite snippets manager:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='python'><span class="n">g</span><span class="o">=</span><span class="nb">globals</span><span class="p">()</span><span class="o">.</span><span class="n">copy</span><span class="p">();</span><span class="n">g</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="nb">locals</span><span class="p">());</span><span class="kn">import</span> <span class="nn">code</span><span class="p">;</span><span class="n">code</span><span class="o">.</span><span class="n">interact</span><span class="p">(</span><span class="n">local</span><span class="o">=</span><span class="n">g</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>


<p>Don’t forget to remove it when you’re done!</p>

<h2>TL;DR</h2>

<ul>
<li>Use <code>virtualenv</code> to isolate your Python environment, <code>pip freeze</code> and a
<code>requirements.txt</code> file to keep track of your dependencies.</li>
<li>Write a <code>pythonrc.py</code> file to add autocomplete support to your interpreter</li>
<li>Use <code>pep8</code>, <code>pylint</code> and <code>pyflakes</code> to keep a high code quality</li>
<li>Use <code>tox</code> to test on multiple Python versions</li>
<li>Fire a local interpreter instead of <code>print</code>ing variables</li>
</ul>


<p>That was all. Please comment on this post if you think of any tool you use to
speed-up your Python development!</p>


feed_id: https://bfontaine.net/blog/2014/09/04/how-to-speed-up-vim-command-t-plugin
title: How to speed-up Vim’s Command-T plugin
uri: 2017-03-05-How-to-speed-up-Vim-s-Command-T-plugin
url: https://bfontaine.net/blog/2014/09/04/how-to-speed-up-vim-command-t-plugin/
content:
<p><a href="https://github.com/wincent/command-t#readme">Command-T</a> is a wonderful Vim plugin which allows you to open files
<q>with a minimal number of keystrokes</q>. It’s really handy in a large
codebase where you only have to type <code>&lt;leader&gt;t</code>, then a couple letters and
press enter to open your file. It’s based on a fuzzy matching, which let you
skip letters without worrying.</p>

<p>I recently installed the plugin on another machine and noticed it was really
low: I had to wait a couple seconds to get the files list everytime. My
computer has 8GB RAM so the problem wasn’t there.</p>

<p>The solution is pretty simple: the plugin relies on a C extension which I
forgot to compile after the installation.</p>

<p>From the docs:</p>

<pre><code>cd ~/.vim/bundle/command-t/ruby/command-t
ruby extconf.rb
make
</code></pre>

<p>It’ll make the plugin incredibly faster. This step is easy to miss if you read
the docs too quickly. I wrote this blog post to remember this, I hope it might
help a couple others since I didn’t find anything on Google about this issue.</p>


title: How to remember the difference between conj and cons in Clojure
uri: 2017-03-05-How-to-remember-the-difference-between-conj-and-cons-in-Clojure
feed_id:
https://bfontaine.net/blog/2014/05/25/how-to-remember-the-difference-between-conj-and-cons-in-clojure

url:
https://bfontaine.net/blog/2014/05/25/how-to-remember-the-difference-between-conj-and-cons-in-clojure/

content:
<p>When I started writing Clojure, I couldn’t memorize the difference between
<code>conj</code> and <code>cons</code> and always used one instead of the another. Their name are
similar, but <code>cons</code> is used to add an element at the beginning of a vector,
while <code>conj</code> is used to add an element at the end of it. How can one memorize
this? I found a mnemonic trick over the time that helps me remember this. Here
is it:</p>

<p>The trick is to look at the last letter of each function, <code>s</code> and <code>j</code>. As shown
in the image below, the <code>s</code> of <code>cons</code> shows the right, while the <code>j</code> of
<code>conj</code> shows the left.</p>

<p><img class="center" src="media/d64b3d3a.sj.png" data-original-source="https://bfontaine.net/blog/images/sj.png"  ></p>

<p>This means that <code>cons</code> pushes elements from the left to the right, that is,
<strong>at the beginning</strong> of a vector. <code>conj</code>, on the other hand, pushes elements
from the right to the left, which is <strong>at the end</strong> of a vector. That’s it.
Once you see this in your head, you’ll never forget the difference between
<code>cons</code> and <code>conj</code> on a vector.</p>


feed_id: https://bfontaine.net/blog/2014/03/03/fixing-gnuplot-pys-unknown-aqua-terminal-warning-on-osx
title: Fixing gnuplot-py’s “unknown aqua terminal” warning on OSX
uri: 2017-03-05-Fixing-gnuplot-py-s-unknown-aqua-terminal-warning-on-OSX
url: https://bfontaine.net/blog/2014/03/03/fixing-gnuplot-pys-unknown-aqua-terminal-warning-on-osx/
content:
<p>When plotting with <a href="https://pypi.python.org/pypi/gnuplot-py"><code>gnuplot-py</code></a> on OSX, I got an annoying warning
saying that terminal <code>aqua</code> is <q>unknown or ambiguous</q>, <em>even</em> when I use a
different terminal (e.g. <code>postscript</code>). This terminal doesn’t exist on my
Gnuplot installation (4.6.3). In fact, <code>gnuplot-py</code> <a href="https://github.com/jrk/gnuplot-py/blob/96b7c7ec81daa560325f4568393e300de020c353/gp.py#L28-39">uses</a> slightly
different files depending on your platform. OSX’s one is exactly the same as
other UNIX-flavored OSes but its default terminal is <code>aqua</code>. There are two ways
to fix the warning, a hacky one I used before this blog post, and a clean one I
discovered while writting this post. Hope this help!</p>

<h2>The hacky way</h2>

<p>Since OSX’s Gnuplot file is the same as Linux’s one apart from its default
term, we just need to tell <code>gnuplot-py</code> we’re on Linux:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='py'><span class="kn">import</span> <span class="nn">sys</span>
<span class="n">_platform</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">platform</span> <span class="c"># save the normal sys.platform value</span>

<span class="k">if</span> <span class="n">_platform</span> <span class="o">==</span> <span class="s">&#39;darwin&#39;</span><span class="p">:</span>
    <span class="n">sys</span><span class="o">.</span><span class="n">platform</span> <span class="o">=</span> <span class="s">&#39;linux&#39;</span> <span class="c"># replace it with &#39;linux&#39;</span>

<span class="kn">import</span> <span class="nn">Gnuplot</span><span class="o">,</span> <span class="nn">Gnuplot.funcutils</span> <span class="c"># import Gnuplot</span>
<span class="n">sys</span><span class="o">.</span><span class="n">platform</span> <span class="o">=</span> <span class="n">_platform</span> <span class="c"># restore it</span>
</code></pre></div></figure></notextile></div>


<p>It temporarily changes <code>sys.platform</code> value, which <code>gnuplot-py</code> <a href="https://github.com/jrk/gnuplot-py/blob/96b7c7ec81daa560325f4568393e300de020c353/gp.py#L28-39">is relying on</a>,
to <code>"linux"</code>, import Gnuplot and then restore it back to its previous value
(<code>"darwin"</code>).</p>

<h2>The clean way</h2>

<p>In fact there’s a much cleaner way to fix the warning. The default terminal is
stored in <a href="https://github.com/jrk/gnuplot-py/blob/96b7c7ec81daa560325f4568393e300de020c353/gp_unix.py#L86-91"><code>GnuplotOpts.default_term</code></a>, so we just need to change
it to another value to fix the warning:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='py'><span class="kn">import</span> <span class="nn">Gnuplot</span><span class="o">,</span> <span class="nn">Gnuplot.funcutils</span>
<span class="n">Gnuplot</span><span class="o">.</span><span class="n">GnuplotOpts</span><span class="o">.</span><span class="n">default_term</span> <span class="o">=</span> <span class="s">&#39;x11&#39;</span>
</code></pre></div></figure></notextile></div>


<p>It’s cleaner than the hacky way because:</p>

<ol>
<li>It’s clear: the second line is pretty explicit: we change <code>default_term</code> to
<code>'x11'</code>. We don’t even need to add a comment.</li>
<li>It’s more maintainable: if OSX’s <code>gnuplot-py</code> interface change in the
future, this piece of code won’t break compatibility while the hacky way
will prevent our code to use the new interface.</li>
<li>It’s shorter: one line versus five ones. It’s really easier to introduce
bugs when using five lines instead of just one.</li>
</ol>


feed_id: https://bfontaine.net/blog/2014/02/15/using-coveralls-with-clojure
title: Using Coveralls with Clojure
uri: 2017-03-05-Using-Coveralls-with-Clojure
url: https://bfontaine.net/blog/2014/02/15/using-coveralls-with-clojure/
content:
<p><a href="https://coveralls.io/"><strong>Coveralls</strong></a> is a service that keep track of your tests coverage
for you. It can notifies you when your coverage decreases under a custom
threshold, and their bot comments on pull requests to report their tests
coverage. Like Travis-CI, it allows you to add a badge to your readme with
an up-to-date tests coverage percentage.</p>

<p>If you already test your GitHub projects with a CI server like Travis, it’s
very easy to add Coveralls to your workflow. Unfortunately, they have a library
for Ruby, a couple user-provided libraries for other languages such as PHP,
Java and Python, but nothing for Clojure. Fortunately, they provide an API for
unsupported languages like Clojure. Here is how to use it.</p>

<h2>Cloverage</h2>

<p><a href="https://github.com/lshift/cloverage"><strong>Cloverage</strong></a> is a Clojure library and a Leiningen plugin to get
form-level tests coverage of your projects. It’s dead easy to use and generates
HTML reports for you as well as some other formats.</p>

<p>Add their plugin to your <code>project.clj</code>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='clj'><span class="p">(</span><span class="kd">defproject </span><span class="nv">your-project</span> <span class="s">&quot;0.1.0&quot;</span>
  <span class="c1">; ...</span>
  <span class="ss">:plugins</span> <span class="p">[[</span><span class="nv">lein-cloverage</span> <span class="s">&quot;1.0.2&quot;</span><span class="p">]])</span>
</code></pre></div></figure></notextile></div>


<p>Use <code>lein cloverage</code> to run your tests and generate various reports. You can
specify which Cloverage version you want to use with <code>CLOVERAGE_VERSION</code>. The
latest stable one is <code>1.0.3</code>, but we’ll need to use <code>1.0.4-SNAPSHOT</code> for this
article:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='sh'><span class="nv">CLOVERAGE_VERSION</span><span class="o">=</span>1.0.4-SNAPSHOT lein cloverage
</code></pre></div></figure></notextile></div>


<h2>Coveralls</h2>

<p>If you don’t already use it, <a href="https://coveralls.io/authorize/github">signup</a> on Coveralls using your GitHub account.
Then, click on “Add Repo”, find your repo in the list and activate it. That’s
all you have to do on their website for now.</p>

<h2>Travis CI</h2>

<p>You now need to custom your builds on Travis to use Coveralls. We’ll do it in a
Bash script, so add this to your <code>.travis.yml</code>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='yaml'><span class="l-Scalar-Plain">after_script</span><span class="p-Indicator">:</span>
  <span class="p-Indicator">-</span> <span class="l-Scalar-Plain">bash -ex test/coveralls.sh</span>
</code></pre></div></figure></notextile></div>


<p>This tells Travis to run <code>test/coveralls.sh</code> after each build. We use Bash
options <code>-e</code> and <code>-x</code> to respectively stop the script at the first failure and
print each command.</p>

<p>Now edit <code>test/coveralls.sh</code> and add the following content:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span class="nv">COVERALLS_URL</span><span class="o">=</span><span class="s1">&#39;https://coveralls.io/api/v1/jobs&#39;</span>
<span class="nv">CLOVERAGE_VERSION</span><span class="o">=</span><span class="s1">&#39;1.0.4-SNAPSHOT&#39;</span> lein2 cloverage -o cov --coveralls
curl -F <span class="s1">&#39;json_file=@cov/coveralls.json&#39;</span> <span class="s2">&quot;$COVERALLS_URL&quot;</span>
</code></pre></div></figure></notextile></div>


<p>It’ll run Cloverage and output reports in <code>cov</code> (use another name if you
prefer), using a report for Coveralls. This generates a <code>coveralls.json</code> file
which can then be sent to their API. That’s what we do on the next line, using
cURL.</p>

<p>That’s all! You can now push on GitHub, Travis will run your tests and send
their tests coverage to Coveralls.</p>

<h2>Caveats</h2>

<p>Coveralls doesn’t support partial-line tests coverage, so we’re cheating a
little bit here using hits count. A line that it never covered has 0 hits, one
that is partially covered has one hit, and a fully covered one has two hits.
There’s <a href="https://github.com/lemurheavy/coveralls-public/issues/216">an open issue</a> regarding this.</p>


feed_id: https://bfontaine.net/blog/2014/02/09/function-level-black-box-testing
title: Function-level Black-box Testing
uri: 2017-03-05-Function-level-Black-box-Testing
url: https://bfontaine.net/blog/2014/02/09/function-level-black-box-testing/
content:
<p>Black-box testing <q>is a method of software testing that examines the
functionality of an application (e.g. what the software does) without peering
into its internal structures or workings</q> (<a href="https://en.wikipedia.org/wiki/Black-box_testing">Wikipedia</a>). While it’s
usually done at a system level, I think the most obvious place it should be
used is at the function level. It’s even more efficient if you write tests for
someone else’s code.
Here is how I write unit tests for functions.</p>

<h2>Good Names</h2>

<p>If you can’t tell what the function roughly does only by looking at its name,
it’s usually because it’s badly named or it’s doing too many things, maybe you
should break it in smaller functions. Look at its arguments names. Do they make
sense to you? Do you <em>understand</em> what’s the function doing only by looking as
its name and its arguments?</p>

<h2>Good Documentation</h2>

<p>You sometimes need to read a function’s documentation if its name is unclear or
if it’s a complicated code area. This is your latest chance to understand the
function, because in function-level black-box testing you can’t read the code,
it would not be black-box anymore. If you understand the function, you can now
write tests. If not, you should improve its name, its arguments names, and its
documentation, in that order because it’s the order one reads it.</p>

<h2>Good Tests</h2>

<p>Good unit tests are short, incremental, and test only a specific case. Start
with the simplest. Does the function takes a string? Give it an empty one. Does
it take a number? Give it <code>0</code>. Use <code>null</code> for objects. Then move to more edge
cases: negative numbers, un-trimmed strings, uninitialized objects. Add more
simple edge cases, but only once at a time. Then, combine those edge cases.
Write more complex cases. Never combine two cases you never individually tested
before. After all these edge cases, test the legitimate ones. Again start with
the simplest, then add more complexity and combine cases.</p>


feed_id: https://bfontaine.net/blog/2013/07/23/open-spotify-links-with-the-desktop-client
title: Open Spotify links with the Desktop client
uri: 2017-03-05-Open-Spotify-links-with-the-Desktop-client
url: https://bfontaine.net/blog/2013/07/23/open-spotify-links-with-the-desktop-client/
content:
<p>Spotify’s <a href="https://play.spotify.com">Web app</a> is great, but you may prefer to
use the desktop client. The problem is that the links default to the Web app.
Here is a quick tip to make what you want.</p>

<p>First, install <a href="https://chrome.google.com/webstore/detail/switcheroo-redirector/cnmciclhnghalnpfhhleggldniplelbg">Switcheroo</a>. It’s a Chrome extension (Firefox users,
go <a href="https://addons.mozilla.org/en-us/firefox/addon/redirector/">here</a>) that allows you
to setup custom redirect rules for any http request, using a string
replacement. Then, add a rule to replace <code>http://open.spotify.com/</code> with
<code>spotify://</code>.</p>

<p>That’s it! Now, all <code>open.spotify.com/something</code> links will open in the desktop
client instead of the Web one.</p>

<p><i class="small">Note: I found the original tip <a href="http://community.spotify.com/t5/Help-Web-Player/PLEASE-allow-users-to-default-to-desktop-client/m-p/460950/highlight/true#M2231">here</a>, but it was not working because the
author suggested to replace with <code>spotify</code> instead of <code>spotify://</code>.</i></p>


feed_id: https://bfontaine.net/blog/2013/06/04/get-the-urban-dictionary-in-your-terminal
title: Get the Urban Dictionary in your terminal
uri: 2017-03-05-Get-the-Urban-Dictionary-in-your-terminal
url: https://bfontaine.net/blog/2013/06/04/get-the-urban-dictionary-in-your-terminal/
content:
<p>The <a href="http://www.urbandictionary.com/">Urban Dictionary</a> website is pretty useful when one wants to
know the meaning of any familiar or slang word. Users can write definitions for
any words, and upvote or downvote them. Unfortunately, the website doesn’t have
a public API, and I couldn’t find a command-line tool for it. So I wrote one.</p>

<h2>Introducing ud</h2>

<p><strong>ud</strong> is a command-line tool which prints definitions from the Urban Dictionary
website. It’s <a href="https://github.com/bfontaine/ud">written</a> in Ruby, and allows you to quickly search for a
definition of any word.</p>

<pre><code>$ ud wth
* WTH (567/126):

   1) Abbreviation for "What the Hell"
   2) Shortened alternative to "With"

 Example:
   1) WTH was that?!
   2) I ate some crackers wth chicken
(…)
</code></pre>

<p>For common abbreviations like “wtf” or “omg”, there are dozens of definitions,
with minor differences. That’s why the script scraps only the first page of
results, and supports a few options: <code>-n</code> allows you to limit the number of
definitions (e.g. <code>-n 1</code> → only one). <code>-r</code> allows you to select only the
definitions which have an upvotes/downvotes ratio higher than what you want,
e.g. <code>-r 2</code> will select only the definitions which have twice more upvotes than
downvotes. For your convenience, it also support the shortcut <code>-u</code> which is an
alias for <code>-r 2</code>. Type <code>ud -h</code> for more info.</p>

<h3>Install</h3>

<p>That’s easy:</p>

<pre><code>gem install ud
</code></pre>

<p>Please note that if you’re using Windows, you’ll need to install the Win32
Console ANSI gem for the colored output. Install it with:</p>

<pre><code>gem install win32console
</code></pre>

<h3>Using the UD module</h3>

<p>The gem also provide a <code>UD</code> module, which you can use to query the website in
your own Ruby code:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='ruby'><span class="nb">require</span> <span class="s1">&#39;ud&#39;</span>

<span class="n">wtf</span> <span class="o">=</span> <span class="no">UD</span><span class="o">.</span><span class="n">query</span> <span class="s1">&#39;wtf&#39;</span><span class="p">,</span> <span class="ss">:count</span> <span class="o">=&gt;</span> <span class="mi">1</span>

<span class="nb">puts</span> <span class="n">wtf</span><span class="o">[</span><span class="mi">0</span><span class="o">].</span><span class="n">keys</span><span class="o">.</span><span class="n">join</span> <span class="s1">&#39;, &#39;</span>
<span class="c1">#=&gt; id, word, definition, example, upvotes, downvotes</span>
</code></pre></div></figure></notextile></div>


feed_id: https://bfontaine.net/blog/2017/04/09/a-quick-link-previewer-for-mattermost
title: A Quick Link Previewer for Mattermost
uri: A-Quick-Link-Previewer-for-Mattermost
url: https://bfontaine.net/blog/2017/04/09/a-quick-link-previewer-for-mattermost/
content:
<p><a href="https://about.mattermost.com/">Mattermost</a> is a <a href="https://slack.com/">Slack</a>-like self-hosted and open-source alternative. We
use it at work but for some reason link previews don’t work. Before diving into
Mattermost’s internals I wanted to see if I could write a quick workaround
using the fact that Mattermost does show an image if you post a link ending
with <code>.png</code> or <code>.jpg</code>.</p>

<h2>The Current Situation</h2>

<p>When you post an image link, Mattermost makes a request to show it in the
application. It detects those images using a <a href="https://github.com/mattermost/platform/blob/3aaf71fdea914af1a7f2b2fb97bb6ae44132fcc4/webapp/components/post_view/components/post_body_additional_content.jsx#L74">regexp</a>; not by e.g.
sending a <code>HEAD</code> request to get the content type. If you have an image URL that
doesn’t end with common extentions Mattermost won’t show it.</p>

<p>Mattermost doesn’t serve you a preview of the image; it rather gives you an
<code>img</code> with the original URL. That means every single person reading the channel
will request the image from its original location. Slack, on the other hand,
fetch images, cache them, and serves them from its own domain,
<code>https://slack-imgs.com</code>. Slack uses a custom user-agent for its request so you
know where it comes from.</p>

<pre><code>User-Agent: Slackbot-LinkExpanding 1.0 (…)
</code></pre>

<p>Mattermost, on the other hand, can’t use a custom user-agent because the
request is done by <em>your</em> browser. The only thing distinguishing Mattermost’s
request for a preview and any other request is it asks for an image:</p>

<pre><code>Accept: image/webp,image/*,*/*;q=0.8
</code></pre>

<p>The header above is Chrome telling the Web server it can deal with <a href="https://developers.google.com/speed/webp/">WebP</a>
images, then images in any format, then anything; in that order. Note
<a href="https://bugs.chromium.org/p/chromium/issues/detail?id=169182">it explicitly says it accepts WebP images</a> because some
browsers don’t support the format.</p>

<p>Unfortunately not all browsers are explicit. Firefox sends <code>Accept: */*</code> since
<a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Content_negotiation/List_of_default_Accept_values">Firefox 47</a> and so did IE8 and earlier versions. In those cases we can’t
really do anything beside complicated rules based on the user-agent and other
headers.</p>

<h2>Proxying Requests</h2>

<p>If we know how to tell if a request comes from Mattermost asking for an image
preview rather than a “normal” user we can serve different contents to them: a
link preview as an image to Mattermost, and the normal content to the user.</p>

<p>All we have to do is to make some sort of intelligent proxy. Using <a href="http://flask.pocoo.org/">Flask</a> we
can make something like this:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='python'><span></span><span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">Flask</span>

<span class="n">app</span> <span class="o">=</span> <span class="n">Flask</span><span class="p">(</span><span class="vm">__name__</span><span class="p">)</span>

<span class="nd">@app.route</span><span class="p">(</span><span class="s2">&quot;/&lt;path:path&gt;/p.png&quot;</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">main</span><span class="p">(</span><span class="n">path</span><span class="p">):</span>
    <span class="k">if</span> <span class="ow">not</span> <span class="n">request</span><span class="o">.</span><span class="n">headers</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;Accept&quot;</span><span class="p">,</span> <span class="s2">&quot;&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">&quot;image/&quot;</span><span class="p">):</span>
        <span class="k">return</span> <span class="n">redirect</span><span class="p">(</span><span class="n">path</span><span class="p">)</span>

    <span class="k">return</span> <span class="s2">&quot;Hello, I&#39;m an image&quot;</span>
</code></pre></div></figure></notextile></div>


<p>This is a small web application that takes any route in the form of
<code>/&lt;url&gt;/p.png</code> and either redirects you to that URL if your <code>Accept</code> header
doesn’t start with <code>image/</code>; either serves you a <code>Hello, I'm an image</code> page.</p>

<p>All we have to do now is to return an actual image in lieu of that placeholder
text. I used <a href="http://docs.python-requests.org/en/master/">Requests</a> and <a href="https://www.crummy.com/software/BeautifulSoup/">Beautiful Soup</a> to fetch and parse
webpages, and <a href="https://pillow.readthedocs.io/en/4.1.x/">Pillow</a> to generate images.</p>

<p>When one requests <code>http://example.com/http://...some...url.../p.png</code>, the
app fetches that URL; parses it to extract its title and some excerpt; write
that on a blank image; and serves it.</p>

<p>Extracting the title is as easy as grabbing that <code>title</code> HTML tag. If
available, I also try the <code>og:title</code> and <code>twitter:title</code> <code>meta</code> tags. If none
of those are available, I fallback on the first <code>h1</code> or <code>h2</code> tag.</p>

<p>Getting an usable excerpt is not too hard; here again I search for common
<code>meta</code> tags: <code>description</code>, <code>og:description</code>, <code>twitter:description</code>. If I can’t
get any of them I take the first <code>p</code> that looks long enough.</p>

<p>You can check that code on <a href="https://github.com/bfontaine/previewer/blob/7e2172de36aa9832bfd71097a760a5d64087dc19/previewer/page.py#L38-L80">GitHub</a>.</p>

<h2>Generating Images</h2>

<p>The Pillow library makes it easy to write text on an image. I replaced the
(ugly) default font with <a href="https://www.fontsquirrel.com/fonts/alegreya">Alegreya</a>. The tricky part is mostly to fit the
text on the image. I used <a href="https://github.com/bfontaine/previewer/blob/7e2172de36aa9832bfd71097a760a5d64087dc19/previewer/text.py">a combination</a> of <a href="https://pillow.readthedocs.io/en/latest/reference/ImageDraw.html#PIL.ImageDraw.PIL.ImageDraw.Draw.textsize">Draw#textsize</a>
to get the dimensions of some text as if it were written on the image and
Python’s <a href="https://docs.python.org/3/library/textwrap.html"><code>textwrap</code></a> module to cut the title and excerpt so that
they wouldn’t cross the right side of the image.</p>

<p>I used fixed dimensions for all images (400×70) and kept a small padding along
their sides. Previews with small or missing excerpts get some unused white
space at the bottom; this could be fixed by pre-computing the final size of the
image <em>before</em> creating it.</p>

<div class='bogus-wrapper'><notextile><figure class="center">
<img class="squared" src="media/4e883eb5.preview-google.png" data-original-source="https://bfontaine.net/blog/images/preview-google.png" alt="Google.com preview" >
<figcaption>Google.com’s generated preview</figcaption>
</figure></notextile></div>




<div class='bogus-wrapper'><notextile><figure class="center">
<img class="squared" src="media/ac397a8a.preview-github.png" data-original-source="https://bfontaine.net/blog/images/preview-github.png" alt="Mattermost's GitHub repo preview" >
<figcaption>GitHub repos have a very small excerpt</figcaption>
</figure></notextile></div>


<p>On most websites the preview works fine. We could tweak the text size as well
as add a favicon or an extracted image.</p>

<div class='bogus-wrapper'><notextile><figure class="center">
<img class="squared" src="media/d357d948.preview-post.png" data-original-source="https://bfontaine.net/blog/images/preview-post.png" alt="This post's preview" >
<figcaption>This post’s generated preview</figcaption>
</figure></notextile></div>


<h2>Conclusion</h2>

<p>In the end it took a couple hours to have a working prototype. Most of that
time was spent dealing with encoding issues and trying various webpages to find
edge cases.</p>

<p>The result is acceptable; it has the issue of not being accessible at all but
that’s still better than nothing.</p>

<p>The source code is on <a href="https://github.com/bfontaine/previewer">GitHub</a>.</p>


feed_id: https://bfontaine.net/blog/2018/11/18/plotting-the-evolution-of-movies-durations-1916-2018
title: Plotting the Evolution of Movies Durations (1916-2018)
uri: Plotting-the-Evolution-of-Movies-Durations-(1916-2018)
url: https://bfontaine.net/blog/2018/11/18/plotting-the-evolution-of-movies-durations-1916-2018/
content:
<p>Two weeks ago I created and put online a <a href="observable">box plot of 73k movies durations
across the time</a>.</p>

<p><img class="center" src="media/3b531966.movies-durations-1916-2018.png" data-original-source="https://bfontaine.net/blog/images/movies-durations-1916-2018.png"  ></p>

<p>It started with a question: “Are movies getting longer and longer?”. Spoiler:
Not really, except maybe in the last couple of years.</p>

<p>I used Wikidata’s online query service to <a href="https://bftn.fr/wikidata-movies-query">export <em>all movies</em></a> then
filtered those with both a publication date and a duration. This gave me a
large JSON which I processed using Python in order to <a href="https://github.com/bfontaine/movies-durations/blob/33194d6e30c263d40ba6f7d7838a42b1c416a5be/movies_durations.py#L28-L43">extract a couple
numbers</a> for each year: min, max, median, first and third quartiles.</p>

<p>The result fits in a <a href="https://gist.github.com/bfontaine/24685b9c193f31d80b7afbcbeb4d0450">small JSON file</a>, which I then used to build a
<a href="https://d3js.org/">D3</a> using a few lines of JS. I used <a href="http://colorbrewer2.org/#type=sequential&amp;scheme=BuGn&amp;n=3">colorbrewer2</a> to find a
colorblind-safe color palette.</p>

<p>You can see the result as well as the JS code <a href="https://beta.observablehq.com/@bfontaine/movies-durations-1916-2018">on Observable</a>.</p>

<p>To avoid outliers such as “<a href="https://en.wikipedia.org/wiki/Modern_Times_Forever_(Stora_Enso_Building,_Helsinki)"><i>Modern Times Forever</i></a>” (240 hours) or
“<a href="https://en.wikipedia.org/wiki/The_Burning_of_the_Red_Lotus_Temple"><i>The Burning of the Red Lotus Temple</i></a>”, I used the <a href="https://en.wikipedia.org/wiki/Interquartile_range#Outliers">interquartile
range</a> (IQR) to limit the size of the bars: any movie whose duration is
below <code>Q1-1.5×IQR</code> or above <code>Q3+1.5×IQR</code> (where <code>Q1</code> is the first quartile and
<code>Q3</code> the third one) is not shown.</p>

<h3>Results</h3>

<p>As one can see on the graph, the median duration quickly rises from 50 to 95
minutes from the 1920s to the 1960s, then doesn’t move much except in the last
two years.</p>

<h3>Limitations</h3>

<p>The first obvious limitation is the data: Wikidata has 200k+ movies but only
73k have both a publication date and a duration. It’s not complete enough to
let me filter by movie type; e.g. feature film vs. others.</p>

<p><a href="https://www.imdb.com/pressroom/stats/">IMDb</a> lists 5.3M titles (most of which are TV episodes), but there’s no way
to export them all.</p>

<p>In the end, there’s no way to know how representative Wikidata’s movies dataset
is. It does give a hint, but this graph is not a definitive answer to the
original question.</p>

<p>See the <a href="https://github.com/bfontaine/movies-durations">Python code</a>.</p>


feed_id: https://bfontaine.net/blog/2018/12/03/make-a-gif-of-a-websites-evolution
title: Make a Gif of a Website’s Evolution
uri: Make-a-Gif-of-a-Website-s-Evolution
url: https://bfontaine.net/blog/2018/12/03/make-a-gif-of-a-websites-evolution/
content:
<p>For the <a href="https://meta.stackexchange.com/q/318910/185171">latest StackExchange “time”-themed contest</a>, I made
<a href="https://meta.stackexchange.com/a/319279/185171">a gif</a> showing the evolution of StackOverflow from 2008 to today:</p>

<p><a href="https://bftn.fr/so-gif"><img src="media/de65c969.so.gif" data-original-source="https://bfontaine.net/so.gif" alt="the gif" /></a></p>

<p class="imghint">(click on the image to play it again)</p>


<p>The first step was to find a decent API for the <a href="https://archive.org/">Internet Archive</a>. It
<a href="https://ws-dl.blogspot.com/2013/07/2013-07-15-wayback-machine-upgrades.html">supports</a> Memento, an HTTP-based protocol defined in the <a href="https://tools.ietf.org/html/rfc7089">RFC 7089</a> in
2013. Using the <a href="https://github.com/mementoweb/py-memento-client#memento-client-"><code>memento_client</code></a> wrapper, we can get the closest snapshot
of a website at a given date with the following Python code:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='python3'><span></span><span class="kn">from</span> <span class="nn">datetime</span> <span class="k">import</span> <span class="n">datetime</span><span class="p">,</span> <span class="n">timedelta</span>
<span class="kn">from</span> <span class="nn">memento_client</span> <span class="k">import</span> <span class="n">MementoClient</span>

<span class="n">mc</span> <span class="o">=</span> <span class="n">MementoClient</span><span class="p">(</span><span class="n">timegate_uri</span><span class="o">=</span><span class="s2">&quot;https://web.archive.org/web/&quot;</span><span class="p">,</span>
        <span class="n">check_native_timegate</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">get_snapshot_url</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">dt</span><span class="p">):</span>
    <span class="n">info</span> <span class="o">=</span> <span class="n">mc</span><span class="o">.</span><span class="n">get_memento_info</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">dt</span><span class="p">)</span>
    <span class="n">closest</span> <span class="o">=</span> <span class="n">info</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;mementos&quot;</span><span class="p">,</span> <span class="p">{})</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;closest&quot;</span><span class="p">)</span>
    <span class="k">if</span> <span class="n">closest</span><span class="p">:</span>
        <span class="k">return</span> <span class="n">closest</span><span class="p">[</span><span class="s2">&quot;uri&quot;</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span>

<span class="c1"># As an example, let’s look at StackOverflow two weeks ago</span>
<span class="n">url</span> <span class="o">=</span> <span class="s2">&quot;https://stackoverflow.com/&quot;</span>
<span class="n">two_weeks_ago</span> <span class="o">=</span> <span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span> <span class="o">-</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">weeks</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
<span class="n">snapshot_url</span> <span class="o">=</span> <span class="n">get_snapshot_url</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">two_weeks_ago</span><span class="p">)</span>

<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;StackOverflow from ~2 weeks ago: </span><span class="si">%s</span><span class="s2">&quot;</span> <span class="o">%</span> <span class="n">snapshot_url</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>


<p>Don’t forget to install the <code>memento_client</code> lib:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span></span>pip install memento_client
</code></pre></div></figure></notextile></div>


<p>Note this gives us the <em>closest</em> snapshot, so it might not be <em>exactly</em> two
weeks ago.</p>

<p>We can use this code to loop using an increasing time delta in order to get
snapshots at different times. But we don’t only want to get the URLs. We wants
to make a screenshot of each one.</p>

<p>The easiest way to programmatically take a screenshot of a webpage is probably
to use Selenium. I used Chrome as a driver; you can either download
it from <a href="https://sites.google.com/a/chromium.org/chromedriver/downloads">the ChromeDriver website</a> or run the following command
if you’re on a Mac with <a href="https://brew.sh/">Homebrew</a>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span></span>brew install bfontaine/utils/chromedriver
</code></pre></div></figure></notextile></div>


<p>We also need to install Selenium for Python:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span></span>pip install selenium
</code></pre></div></figure></notextile></div>


<p>The code is pretty short:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='python3'><span></span><span class="kn">from</span> <span class="nn">selenium</span> <span class="k">import</span> <span class="n">webdriver</span>

<span class="k">def</span> <span class="nf">make_screenshot</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">filename</span><span class="p">):</span>
    <span class="n">driver</span> <span class="o">=</span> <span class="n">webdriver</span><span class="o">.</span><span class="n">Chrome</span><span class="p">(</span><span class="s2">&quot;chromedriver&quot;</span><span class="p">)</span>
    <span class="n">driver</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
    <span class="n">driver</span><span class="o">.</span><span class="n">save_screenshot</span><span class="p">(</span><span class="n">filename</span><span class="p">)</span>
    <span class="n">driver</span><span class="o">.</span><span class="n">quit</span><span class="p">()</span>

<span class="n">url</span> <span class="o">=</span> <span class="s2">&quot;https://web.archive.org/web/20181119211854/https://stackoverflow.com/&quot;</span>

<span class="n">make_screenshot</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="s2">&quot;stackoverflow_20181119211854.png&quot;</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>


<p>If you run the code above, you should see a Chrome window open, go at the URL
by itself, then close once the page is fully charged. You now have a screenshot
of this page in <code>stackoverflow_20181119211854.png</code>! However, you’ll quickly
notice the screenshot includes the Wayback Machine’s header over the top of the
website:</p>

<p><img class="center" src="media/40dead4f.stackoverflow_20181119211854_top.png" data-original-source="https://bfontaine.net/blog/images/stackoverflow_20181119211854_top.png"  ></p>

<p>This is handy when browsing through snapshots by hand, but not so much when we
access them from Python.</p>

<p>Fortunately, we can get a header-less URL by <a href="https://webapps.stackexchange.com/a/40912/77124">changing it a bit</a>: we can
append <code>id_</code> to the end of the date in order to get the page exactly as it was
when the bot crawled it. However, this means it links to CSS and JS files that
may not exist anymore. We can get a URL to an archived page that has been
slightly modified to replace links with their archived version using <code>im_</code>
instead.</p>

<dl>
<dt>Page with header and rewritten links:</dt>
<dd><code>https://web.archive.org/web/20181119211854/...</code></dd>
<dt>Original page, as it was when crawled:</dt>
<dd><code>https://web.archive.org/web/20181119211854<b>id_</b>/...</code></dd>
<dt>Original page with rewritten links:</dt>
<dd><code>https://web.archive.org/web/20181119211854<b>im_</b>/...</code></dd>
</dl>


<p>Re-running the code using the modified URL gives us the correct screenshot:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='python'><span></span><span class="n">url</span> <span class="o">=</span> <span class="s2">&quot;https://web.archive.org/web/20181119211854im_/https://stackoverflow.com/&quot;</span>
<span class="n">make_screenshot</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="s2">&quot;stackoverflow_20181119211854.png&quot;</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>


<p>Joining the two bits of code we can make screenshots of a URL at different
intervals. You may want to check the images once it’s done to remove
inconsistencies. For example, the archived snapshots of Google’s homepage
aren’t all in the same language.</p>

<p>Once we have all images, we can generate a gif using <a href="https://www.imagemagick.org/script/index.php">Imagemagick</a>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='bash'><span></span>convert -delay <span class="m">50</span> -loop <span class="m">1</span> *.png stackoverflow.gif
</code></pre></div></figure></notextile></div>


<p>I used the following parameters:</p>

<ul>
<li><code>-delay 50</code>: change frame every 0.5s. The number is in 100th of a second.</li>
<li><code>-loop 1</code>: loop only once over all the frames. The default is to make an
infinite loop but it doesn’t really make sense here.</li>
</ul>


<p>You may want to play with the <code>-delay</code> parameter depending on how many images
you have as well as how often the website changes.</p>

<p>I also made <a href="media/aa2321c8.google.gif" data-original-source="https://bfontaine.net/google.gif">a version with Google</a> (~10MB) at 5 frames per second,
with <code style="white-space:nowrap">-delay 20</code>. I used the same delay
as the StackOverflow gif: at least 5&nbsp;weeks between each screenshot. You
can see which year the screenshot is from by looking at the bottom of each
image.</p>


feed_id: https://bfontaine.net/blog/2019/12/01/introduction-to-code-golf-in-clojure
title: Introduction to Code-Golf in Clojure
uri: Introduction-to-Code-Golf-in-Clojure
url: https://bfontaine.net/blog/2019/12/01/introduction-to-code-golf-in-clojure/
content:
<p>Code-Golf is the art of writing the shortest program in a given language that
implements some given algorithm. It started in the 90’s in the Perl community
and spread to other languages; there are now languages dedicated to
code-golfing and StackExchange has a <a href="https://codegolf.stackexchange.com/">Q&amp;A website</a> for it.</p>

<p>In 2015, for example, I wrote a blog post showing how to write a <a href="/2015/10/29/a-javascript-modules-manager-that-fits-in-a-tweet/">JavaScript modules manager that fits in 140 chars</a> (the maximum length
of a tweet at that time).</p>

<p><a href="http://www.4clojure.com/">4clojure</a> is a well-known website to learn Clojure
through exercises of increasing difficulty, but it has a lesser-known code-golf
challenge which you can enable by clicking on “<a href="http://www.4clojure.com/league">Leagues</a>” in the top menu.
If you check the code-golf checkbox, you then get a score on each problem that
is the number of non-whitespace characters of your solution; the smaller the
better.</p>

<p>The first thing you’ll note when code-golfing is that the reader syntax for
anonymous functions is a lot shorter than using <code>fn</code>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='clojure'><span></span><span class="c1">; 18 chars</span>
<span class="p">(</span><span class="k">fn </span><span class="p">[</span><span class="nv">a</span> <span class="nv">b</span> <span class="nv">c</span><span class="p">]</span> <span class="p">(</span><span class="nb">* </span><span class="p">(</span><span class="nb">+ </span><span class="nv">a</span> <span class="nv">b</span><span class="p">)</span> <span class="nv">c</span><span class="p">))</span>

<span class="c1">; 13 chars</span>
<span class="o">#</span><span class="p">(</span><span class="nb">* </span><span class="p">(</span><span class="nb">+ </span><span class="nv">%1</span> <span class="nv">%2</span><span class="p">)</span> <span class="nv">%3</span><span class="p">)</span>
<span class="c1">; 12 chars: -1 char because &#39;%&#39; is equivalent to &#39;%1&#39;</span>
<span class="o">#</span><span class="p">(</span><span class="nb">* </span><span class="p">(</span><span class="nb">+ </span><span class="nv">%</span> <span class="nv">%2</span><span class="p">)</span> <span class="nv">%3</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>


<p>Unfortunately you can’t have a reader-syntax function inside another
reader-syntax one, so you often have to transform the code not to use anonymous
functions.</p>

<p><code>for</code> is a very powerful tool for that, because it allows you to do the
equivalent of <code>map</code>, and a lot more, with no function:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='clojure'><span></span><span class="c1">; invalid!</span>
<span class="o">#</span><span class="p">(</span><span class="nb">map </span><span class="o">#</span><span class="p">(</span><span class="nb">* </span><span class="mi">2</span> <span class="nv">%</span><span class="p">)</span> <span class="nv">%</span><span class="p">)</span>

<span class="c1">; 19 chars</span>
<span class="o">#</span><span class="p">(</span><span class="nb">map </span><span class="p">(</span><span class="k">fn </span><span class="p">[</span><span class="nv">x</span><span class="p">]</span> <span class="p">(</span><span class="nb">* </span><span class="mi">2</span> <span class="nv">x</span><span class="p">))</span> <span class="nv">%</span><span class="p">)</span>
<span class="c1">; 17 chars</span>
<span class="o">#</span><span class="p">(</span><span class="nb">map </span><span class="p">(</span><span class="nb">partial * </span><span class="mi">2</span><span class="p">)</span> <span class="nv">%</span><span class="p">)</span>
<span class="c1">; 15 chars</span>
<span class="o">#</span><span class="p">(</span><span class="nb">for </span><span class="p">[</span><span class="nv">x</span> <span class="nv">%</span><span class="p">]</span> <span class="p">(</span><span class="nb">* </span><span class="mi">2</span> <span class="nv">x</span><span class="p">))</span>

<span class="c1">; Note that for this specific example</span>
<span class="c1">; the best solution uses `map`:</span>
<span class="o">#</span><span class="p">(</span><span class="nb">map + </span><span class="nv">%</span> <span class="nv">%</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>


<p>On some problems it can even be shorter than using <code>map</code> + <code>filter</code>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='clojure'><span></span><span class="c1">; 31 chars</span>
<span class="p">(</span><span class="k">fn </span><span class="p">[</span><span class="nv">x</span> <span class="nv">a</span><span class="p">]</span>
  <span class="p">(</span><span class="nb">map inc </span><span class="p">(</span><span class="nb">filter </span><span class="o">#</span><span class="p">(</span><span class="nb">&lt; </span><span class="nv">%</span> <span class="nv">a</span><span class="p">)</span> <span class="nv">x</span><span class="p">)))</span>

<span class="c1">; 26 chars</span>
<span class="o">#</span><span class="p">(</span><span class="nb">for </span><span class="p">[</span><span class="nv">e</span> <span class="nv">x</span> <span class="ss">:when</span> <span class="p">(</span><span class="nb">&lt; </span><span class="nv">e</span> <span class="nv">a</span><span class="p">)]</span> <span class="p">(</span><span class="nb">inc </span><span class="nv">e</span><span class="p">))</span>
</code></pre></div></figure></notextile></div>


<p>Some core functions are equivalent in some contexts and so the shorter one can
substitute a longer one:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='clojure'><span></span><span class="c1">; 18 chars</span>
<span class="o">#</span><span class="p">(</span><span class="nb">filter identity </span><span class="nv">%</span><span class="p">)</span>
<span class="c1">; 14 chars</span>
<span class="o">#</span><span class="p">(</span><span class="nb">filter comp </span><span class="nv">%</span><span class="p">)</span>

<span class="c1">; 6 chars</span>
<span class="p">(</span><span class="nb">inc </span><span class="nv">x</span><span class="p">)</span>
<span class="p">(</span><span class="nb">dec </span><span class="nv">x</span><span class="p">)</span>
<span class="c1">; 5 chars</span>
<span class="p">(</span><span class="nb">+ </span><span class="nv">x</span> <span class="mi">1</span><span class="p">)</span>
<span class="p">(</span><span class="nb">- </span><span class="nv">x</span> <span class="mi">1</span><span class="p">)</span>

<span class="c1">; 12 chars</span>
<span class="p">(</span><span class="nb">reduce str </span><span class="nv">x</span><span class="p">)</span>
<span class="c1">; 11 chars</span>
<span class="p">(</span><span class="nb">apply str </span><span class="nv">x</span><span class="p">)</span>

<span class="c1">; 14 chars</span>
<span class="p">(</span><span class="nb">apply concat </span><span class="nv">x</span><span class="p">)</span>
<span class="c1">; 13 chars</span>
<span class="p">(</span><span class="nb">mapcat comp </span><span class="nv">x</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>


<p>When you must use a long function name in multiple places, it might be shorter
to <code>let</code> that function with a one-letter symbol:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='clojure'><span></span><span class="c1">; 120 chars</span>
<span class="o">#</span><span class="p">(</span><span class="nf">clojure.set/difference</span>
   <span class="p">(</span><span class="nf">clojure.set/union</span> <span class="nv">%</span> <span class="nv">%2</span><span class="p">)</span>
   <span class="p">(</span><span class="nf">clojure.set/union</span>
     <span class="p">(</span><span class="nf">clojure.set/difference</span> <span class="nv">%</span> <span class="nv">%2</span><span class="p">)</span>
     <span class="p">(</span><span class="nf">clojure.set/difference</span> <span class="nv">%2</span> <span class="nv">%</span><span class="p">)))</span>

<span class="c1">; 73 chars</span>
<span class="o">#</span><span class="p">(</span><span class="k">let </span><span class="p">[</span><span class="nv">d</span> <span class="nv">clojure.set/difference</span> <span class="nv">u</span> <span class="nv">clojure.set/union</span><span class="p">]</span>
   <span class="p">(</span><span class="nf">d</span> <span class="p">(</span><span class="nf">u</span> <span class="nv">%</span> <span class="nv">%2</span><span class="p">)</span> <span class="p">(</span><span class="nf">u</span> <span class="p">(</span><span class="nf">d</span> <span class="nv">%</span> <span class="nv">%2</span><span class="p">)</span> <span class="p">(</span><span class="nf">d</span> <span class="nv">%2</span> <span class="nv">%</span><span class="p">))))</span>

<span class="c1">; Note that for this specific example</span>
<span class="c1">; there is a 17-chars solution</span>
<span class="o">#</span><span class="p">(</span><span class="nb">set </span><span class="p">(</span><span class="nb">filter </span><span class="nv">%2</span> <span class="nv">%</span><span class="p">))</span>
</code></pre></div></figure></notextile></div>


<h3>Other tricks</h3>

<p>Use indexed access on vectors:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='clojure'><span></span><span class="c1">; 15 chars</span>
<span class="p">(</span><span class="nb">first </span><span class="p">[</span><span class="ss">:a</span> <span class="ss">:b</span> <span class="ss">:c</span><span class="p">])</span>
<span class="c1">; 11 chars</span>
<span class="p">([</span><span class="ss">:a</span> <span class="ss">:b</span> <span class="ss">:c</span><span class="p">]</span> <span class="mi">0</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>


<p>Use set literals as functions:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='clojure'><span></span><span class="c1">; 16 chars</span>
<span class="p">(</span><span class="nb">remove </span><span class="o">#</span><span class="p">(</span><span class="nb">= </span><span class="ss">:a</span> <span class="nv">%</span><span class="p">)</span> <span class="nv">x</span><span class="p">)</span>
<span class="c1">; 14 chars</span>
<span class="p">(</span><span class="nb">remove </span><span class="o">#</span><span class="p">{</span><span class="ss">:a</span><span class="p">}</span> <span class="nv">x</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>


<p>Inverse conditions to use shorter functions:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='clojure'><span></span><span class="c1">; 15 chars</span>
<span class="p">(</span><span class="k">if </span><span class="p">(</span><span class="nf">empty?</span> <span class="nv">p</span><span class="p">)</span> <span class="nv">a</span> <span class="nv">b</span><span class="p">)</span>
<span class="c1">; 12 chars</span>
<span class="p">(</span><span class="k">if </span><span class="p">(</span><span class="nb">seq </span><span class="nv">p</span><span class="p">)</span> <span class="nv">b</span> <span class="nv">a</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>


<p>Inlined code is sometimes shorter:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='clojure'><span></span><span class="c1">; 24 chars</span>
<span class="p">(</span><span class="k">let </span><span class="p">[</span><span class="nv">p</span> <span class="p">(</span><span class="nb">* </span><span class="mi">3</span> <span class="nv">a</span><span class="p">)]</span>
  <span class="p">(</span><span class="k">if </span><span class="p">(</span><span class="nb">&lt; </span><span class="nv">p</span> <span class="mi">5</span><span class="p">)</span>
    <span class="nv">a</span>
    <span class="nv">p</span><span class="p">))</span>
<span class="c1">; 19 chars</span>
<span class="p">(</span><span class="k">if </span><span class="p">(</span><span class="nb">&lt; </span><span class="p">(</span><span class="nb">* </span><span class="mi">3</span> <span class="nv">a</span><span class="p">)</span> <span class="mi">5</span><span class="p">)</span>
  <span class="nv">a</span>
  <span class="p">(</span><span class="nb">* </span><span class="mi">3</span> <span class="nv">a</span><span class="p">))</span>
</code></pre></div></figure></notextile></div>


<p>Use <code>1</code> instead of <code>:else</code>/<code>:default</code> in <code>cond</code>:</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='clojure'><span></span><span class="c1">; 24 chars</span>
<span class="p">(</span><span class="nf">cond</span>
  <span class="p">(</span><span class="nb">= </span><span class="nv">m</span> <span class="nv">p</span><span class="p">)</span> <span class="nv">a</span>
  <span class="p">(</span><span class="nb">&lt; </span><span class="nv">m</span> <span class="nv">p</span><span class="p">)</span> <span class="nv">b</span>
  <span class="ss">:else</span> <span class="nv">c</span><span class="p">)</span>

<span class="c1">; 20 chars</span>
<span class="p">(</span><span class="nf">cond</span>
  <span class="p">(</span><span class="nb">= </span><span class="nv">m</span> <span class="nv">p</span><span class="p">)</span> <span class="nv">a</span>
  <span class="p">(</span><span class="nb">&lt; </span><span class="nv">m</span> <span class="nv">p</span><span class="p">)</span> <span class="nv">b</span>
  <span class="mi">1</span> <span class="nv">c</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>


<p>Use maps instead of <code>if</code>s for conditions on equality (this one <em>really</em> makes
the&nbsp;code harder to read):</p>

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span></span></figcaption><div class="highlight"><pre><code class='clojure'><span></span><span class="c1">; 13 chars</span>
<span class="p">(</span><span class="k">if </span><span class="p">(</span><span class="nb">= </span><span class="s">&quot;L&quot;</span> <span class="nv">x</span><span class="p">)</span> <span class="nv">a</span> <span class="nv">b</span><span class="p">)</span>
<span class="c1">; 12 chars</span>
<span class="p">(</span><span class="nf">case</span> <span class="nv">x</span> <span class="s">&quot;L&quot;</span> <span class="nv">a</span> <span class="nv">b</span><span class="p">)</span>
<span class="c1">; 10 chars</span>
<span class="p">({</span><span class="s">&quot;L&quot;</span> <span class="nv">a</span><span class="p">}</span> <span class="nv">x</span> <span class="nv">b</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>


feed_id: https://bfontaine.net/blog/2020/11/20/fix-virtualbox-macos
title: Fix Virtualbox installation for Docker on macOS
uri: Fix-Virtualbox-installation-for-Docker-on-macOS
url: https://bfontaine.net/blog/2020/11/20/fix-virtualbox-macos/
content:
<p>While following a <a href="https://medium.com/crowdbotics/a-complete-one-by-one-guide-to-install-docker-on-your-mac-os-using-homebrew-e818eb4cfc3">tutorial</a> to install Virtualbox in order to have <code>docker</code> working on macOS, I
hit an issue where the <code>docker-machine create</code> command fails with an error that looks like this:</p>

<div class="bogus-wrapper"><notextile><figure class="code"><div class="highlight"><pre><code class="text"><span></span>VBoxManage: error: Failed to create the host-only adapter
VBoxManage: error: VBoxNetAdpCtl: Error while adding new interface: failed to open /dev/vboxnetctl: No such file or directory
VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component HostNetworkInterfaceWrap, interface IHostNetworkInterface
VBoxManage: error: Context: &quot;RTEXITCODE handleCreate(HandlerArg *)&quot; at line 95 of file VBoxManageHostonly.cpp
</code></pre></div></figure></notextile></div>

<p>If you search on the Web, everybody says you have to open the Security &amp; Privacy settings window
and allow the Oracle kernel extensions to run. But I didn’t have it. I tried uninstalling
Virtualbox, re-installing through the official website, reboot, uninstall, re-install with
<code>brew cask</code> but I always had the issue. <a href="http://biercoff.com/how-to-fix-docker-machine-installation-on-mac-os-x/">Some people</a> reported having a failed Virtualbox
installation but mine seemed ok.</p>

<p>I tried <a href="https://forums.virtualbox.org/viewtopic.php?f=8&amp;t=88142#p428629">the <code>spctl</code> solution</a> but it didn’t change anything.</p>

<p>In the end, I tried <a href="https://stackoverflow.com/a/59961248/735926">this StackOverflow answer</a>:</p>

<div class="bogus-wrapper"><notextile><figure class="code"><figcaption><span></span></figcaption><div class="highlight"><pre><code class="bash"><span></span>sudo <span class="s2">&quot;/Library/Application Support/VirtualBox/LaunchDaemons/VirtualBoxStartup.sh&quot;</span> restart
</code></pre></div></figure></notextile></div>

<p>It failed, but it told me to check the Security &amp; Privacy setting window. I did, and I had the
button everyone was talking about. I enabled the kernel extension, rebooted, and it worked.</p>

<p>Hope this can save some time to anyone having the same issue!</p>


feed_id: https://bfontaine.net/blog/2020/12/16/fix-bin-magento-all-the-ram
title: Fix bin/magento taking all the RAM
uri: Fix-bin-magento-taking-all-the-RAM
url: https://bfontaine.net/blog/2020/12/16/fix-bin-magento-all-the-ram/
content:
<p>While working with Magento 2.3.6 on <a href="https://bixoto.com/">Bixoto</a> I hit a weird issue: the
<code>bin/magento</code> command-line tool was always eating all the RAM, even with a simple command:</p>

<div class="bogus-wrapper"><notextile><figure class="code"><div class="highlight"><pre><code class="text"><span></span>$ ./bin/magento --help
PHP Fatal error:  Allowed memory size of 2147483648 bytes exhausted (tried to allocate 262144 bytes) in /home/baptiste/.../vendor/magento/module-store/Model/Config/Placeholder.php on line 146
Check https://getcomposer.org/doc/articles/troubleshooting.md#memory-limit-errors for more info on how to handle out of memory errors.
</code></pre></div></figure></notextile></div>

<p>The issue, as weird as it sounds, is an empty configuration value that causes Magento to end up in
an infinite loop.</p>

<p>When I installed Magento on my local machine, I deactivated HTTPS by setting <code>web/secure/base_url</code>
to <code>NULL</code> in the table <code>core_config_data</code>. This alone is the cause of the issue.</p>

<p>Check in MySQL:</p>

<div class="bogus-wrapper"><notextile><figure class="code"><figcaption><span></span></figcaption><div class="highlight"><pre><code class="sql"><span></span><span class="k">SELECT</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">core_config_data</span> <span class="k">WHERE</span> <span class="n">path</span> <span class="o">=</span> <span class="s1">&#39;web/secure/base_url&#39;</span> <span class="k">LIMIT</span> <span class="mi">1</span><span class="p">;</span>
</code></pre></div></figure></notextile></div>

<p>If this shows a line with a <code>NULL</code> value, either delete it or replace it with some non-null value:</p>

<div class="bogus-wrapper"><notextile><figure class="code"><figcaption><span></span></figcaption><div class="highlight"><pre><code class="sql"><span></span><span class="k">UPDATE</span> <span class="n">core_config_data</span> <span class="k">SET</span> <span class="n">value</span><span class="o">=</span><span class="s1">&#39;http://...&#39;</span> <span class="k">WHERE</span> <span class="n">path</span> <span class="o">=</span> <span class="s1">&#39;web/secure/base_url&#39;</span> <span class="k">limit</span> <span class="mi">1</span><span class="p">;</span>
</code></pre></div></figure></notextile></div>

<p>This has been <a href="https://github.com/magento/magento2/issues/12835">reported to Magento</a> but was closed because “it’s not a bug”. I don’t think
falling in an infinite loop on <code>--help</code> because some config value is <code>NULL</code> should really be a
normal behavior, but at least now you know how to solve it.</p>


feed_id: https://bfontaine.net/blog/2021/09/25/typing-tuples-in-python
title: Typing Tuples in Python
uri: Typing-Tuples-in-Python
url: https://bfontaine.net/blog/2021/09/25/typing-tuples-in-python/
content:
<p>Python added support for type hints in 3.5. These are not typing as you may be used to in other languages
since they have no effect on the compilation to bytecode nor at runtime, but rather hints for the
(tools of the) developer.</p>

<div class="bogus-wrapper"><notextile><figure class="code"><figcaption><span></span></figcaption><div class="highlight"><pre><code class="python3"><span></span><span class="k">def</span> <span class="nf">print_int</span><span class="p">(</span><span class="n">n</span><span class="p">:</span> <span class="nb">int</span><span class="p">):</span>
    <span class="nb">print</span><span class="p">(</span><span class="n">n</span><span class="p">)</span>

<span class="n">print_int</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="n">print_int</span><span class="p">(</span><span class="s2">&quot;foo&quot;</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>

<p>The code above doesn’t fail when you call <code>print_int("foo")</code> even though <code>n</code> is “typed” as an <code>int</code>. This is because
this <code>n: int</code> is just a hint.</p>

<p>While you can check for type issues by running <a href="http://mypy-lang.org/"><code>mypy</code></a> by hand, type hints become really
powerful when your editor/IDE supports them.</p>

<p>Types for collections can specify the inner types: a list (<code>List</code>) that contains strings (<code>str</code>) would be <code>List[str]</code>.</p>

<div class="bogus-wrapper"><notextile><figure class="code"><figcaption><span></span></figcaption><div class="highlight"><pre><code class="python3"><span></span><span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">List</span>

<span class="n">stuff</span><span class="p">:</span> <span class="nb">list</span> <span class="o">=</span> <span class="p">[]</span>  <span class="c1"># a list of anything (equivalent to List[Any])</span>
<span class="n">stuff</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="n">stuff</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">&quot;foo&quot;</span><span class="p">)</span>

<span class="n">offsets</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="nb">int</span><span class="p">]</span> <span class="o">=</span> <span class="p">[]</span>  <span class="c1"># a list of ints</span>
<span class="n">offsets</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="n">offsets</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">&quot;foo&quot;</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>

<p>In the snippet above, the last line is highlighted as an error in any good IDE, and <code>mypy</code> would complain about it.</p>

<p>Other container types exist as well, and they can be nested:</p>

<div class="bogus-wrapper"><notextile><figure class="code"><figcaption><span></span></figcaption><div class="highlight"><pre><code class="python3"><span></span><span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">List</span><span class="p">,</span> <span class="n">Dict</span><span class="p">,</span> <span class="n">Iterable</span>

<span class="c1"># a dictionary where keys are str and values are List[str], i.e. lists of strings</span>
<span class="n">friends</span><span class="p">:</span> <span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">friends</span><span class="p">[</span><span class="s2">&quot;Alice&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;Sam&quot;</span><span class="p">,</span> <span class="s2">&quot;Maria&quot;</span><span class="p">]</span>

<span class="c1"># a function that takes an iterable of ints. It can be a list, a tuple, a generator, a set, etc</span>
<span class="k">def</span> <span class="nf">average</span><span class="p">(</span><span class="n">s</span><span class="p">:</span> <span class="n">Iterable</span><span class="p">[</span><span class="nb">int</span><span class="p">]):</span>
    <span class="n">total</span> <span class="o">=</span> <span class="mi">0</span>
    <span class="n">count</span> <span class="o">=</span> <span class="mi">0</span>
    <span class="k">for</span> <span class="n">element</span> <span class="ow">in</span> <span class="n">s</span><span class="p">:</span>
        <span class="n">total</span> <span class="o">+=</span> <span class="n">element</span>
        <span class="n">count</span> <span class="o">+=</span> <span class="mi">1</span>

    <span class="k">return</span> <span class="n">total</span> <span class="o">/</span> <span class="n">count</span>
</code></pre></div></figure></notextile></div>

<p>Given <code>List[x]</code>, <code>Collection[x]</code>, <code>Sequence[x]</code> and other <code>Set[x]</code>, one would expect <code>Tuple[x]</code> to be a hint for a tuple
that contains <code>x</code>. Well… no.</p>

<p>This is confusing at first, but <code>Tuple[str]</code> types a tuple of <u>a&nbsp;single&nbsp;element</u> of type <code>str</code>. To add
more elements, you need to type them as well: a pair of ints would be <code>Tuple[int, int]</code>, while a triplet of a string,
a float and a boolean would be <code>Tuple[str, int, bool]</code>.</p>

<p>While tuples can be used as sequences (e.g. for immutable/hashable equivalents to lists), I’d argue that their primary
use is for fixed-length representations, such as pairs of results:</p>

<div class="bogus-wrapper"><notextile><figure class="code"><figcaption><span></span></figcaption><div class="highlight"><pre><code class="python3"><span></span><span class="k">def</span> <span class="nf">match_object</span><span class="p">(</span><span class="n">data</span><span class="p">:</span> <span class="nb">bytes</span><span class="p">):</span>
    <span class="c1"># example code</span>
    <span class="n">distances</span> <span class="o">=</span> <span class="p">[</span><span class="mf">0.99</span><span class="p">,</span> <span class="mf">0.97</span><span class="p">,</span> <span class="mf">0.96</span><span class="p">]</span>
    <span class="n">indices</span> <span class="o">=</span> <span class="p">[</span><span class="mi">432</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">3923</span><span class="p">]</span>
    <span class="k">return</span> <span class="n">distances</span><span class="p">,</span> <span class="n">indices</span>
</code></pre></div></figure></notextile></div>

<p>In this snippet, <code>match_object</code> returns a tuple of a list of floats and a list of integers
(aka <code>Tuple[List[float], List[int]]</code>).</p>

<p>If you still want to type arbitrary-length homogeneous tuples, there’s a syntax for that: <code>Tuple[int, ...]</code>
types a tuple of any length, including 0, that contains only <code>int</code> elements (and yes, <code>...</code> is valid in Python).</p>

<p>For this and other interrogations (how to type a generator?), Mypy has a useful <a href="https://mypy.readthedocs.io/en/latest/cheat_sheet_py3.html">type hints cheat sheet</a>.</p>

<p><strong>TL;DR</strong>: if you know the size of the tuple, use <code>Tuple[x]</code>, <code>Tuple[x, x]</code>, <code>Tuple[x, x, x]</code>, etc. If you don’t,
use <code>Tuple[x, ...]</code>, but all elements must be of type <code>x</code>.</p>


feed_id: https://bfontaine.net/blog/2022/08/23/unique-strings-and-unicode
title: Unique Strings and Unicode
uri: Unique-Strings-and-Unicode
url: https://bfontaine.net/blog/2022/08/23/unique-strings-and-unicode/
content:
<p>For the past three years I’ve been running a Twitter bot called <a href="https://twitter.com/lemondeadit">@lemondeadit</a>, a French equivalent of
<a href="https://twitter.com/NYT_first_said">@NYT_first_said</a> which tweets new words as they appear in the French newspaper <i>Le Monde</i>.</p>

<p>The account was created in June 2019 but the first commit dates back to 2018. One day I might blog a bit about how it
works under the hood, but today I’d like to talk about a small bug that made me crazy for a while.</p>

<p>The bot has been running stable for a couple of years without me touching the code at all. But recently, I noticed that
sometimes it would tweet some very common words as if they were new, like “<i lang="fr">économie</i>” or
“<i lang="fr">opéra</i>”. I deleted them when I saw them, but I didn’t understand why it happened.</p>

<p>I initially attributed the bug to an issue with the search engine: when the bot sees a word it never saw before, it uses
<i>Le Monde</i>’s internal search engine to verify that this word indeed appears in a single article. Around the end of
2019,<i>Le Monde</i> ditched their previous search engine, which was slow but exact, in favor of a Qwant-based one which
is fast but inexact: it doesn’t respect your query (it may or may not autocorrect your query and there’s no way to prevent
it from doing so), it’s not accurate (no result may or may not mean that there is no match) and it’s not stable (do the
same query twice and you can get different results; even by a large order of magnitude). In French, we would say this is
“<i lang="fr">de la merde</i>”.</p>

<p>Anyway, I thought it was an issue where maybe using the internal search engine would give a single result instead of
multiple ones, but this doesn’t make sense: the bot doesn’t check every single word with the search engine, but only
those it thinks are unique. Since it has already scrapped virtually all articles from the newspaper, there’s no way it
would consider “<i lang="fr">économie</i>” as a new word.</p>

<p>Today, I decided to tackle the issue.</p>

<p>First, I searched in the database for a recent example, and I got “<i lang="fr">vaccinées</i>” (the feminin plural form
of “vaccinated”). This is a very old word; it first appeared in <i>Le Monde</i> in <a href="https://www.lemonde.fr/archives/article/1946/10/04/la-crise-signalee-dans-les-arrivages-de-lait-a-paris-semble-terminee_1868181_1819218.html#:~:text=vaccin%C3%A9es">1946</a> and we saw it more than
1.7k times since then. The word even showed up twice in the database, despite a <code>UNIQUE</code> constraint on the field:</p>

<div class="bogus-wrapper"><notextile><figure class="code"><figcaption><span></span></figcaption><div class="highlight"><pre><code class="sql"><span></span>   <span class="n">word</span>
<span class="c1">-----------</span>
 <span class="n">vaccine</span><span class="err">́</span><span class="n">es</span>
 <span class="n">vaccinées</span>
</code></pre></div></figure></notextile></div>

<p>I inspected the strings and noticed they were encoded differently, although they were both in UTF-8. The difference was
on the “é” character: it turns out that both are <a href="https://apps.timwhitlock.info/unicode/inspect?s=%C3%A9e%CC%81">different</a>: one is represented as the Unicode character
<code>LATIN SMALL LETTER E WITH ACUTE</code> while the other is a combination of <code>LATIN SMALL LETTER E</code> and
<code>COMBINING ACUTE ACCENT</code>. They both render <em>exactly</em> the same on screen, which makes this issue so hard to detect by a
human. Another variant of this issue is with characters that have different codepoints but exactly the same rendering,
like “Ⅰ” (<code>ROMAN NUMERAL ONE</code>) and “I” (<code>LATIN CAPITAL LETTER I</code>).</p>

<div class="bogus-wrapper"><notextile><figure class="code"><figcaption><span></span></figcaption><div class="highlight"><pre><code class="pycon"><span></span><span class="gp">&gt;&gt;&gt; </span><span class="s2">&quot;é&quot;</span> <span class="o">==</span> <span class="s2">&quot;é&quot;</span>
<span class="go">False</span>
<span class="gp">&gt;&gt;&gt; </span><span class="s2">&quot;Ⅰ&quot;</span> <span class="o">==</span> <span class="s2">&quot;I&quot;</span>
<span class="go">False</span>
</code></pre></div></figure></notextile></div>

<p>The fix was to normalize the text before interpreting its words. I did it in Python using <a href="https://docs.python.org/3/library/unicodedata.html#unicodedata.normalize"><code>unicodedata.normalize</code></a>:</p>

<div class="bogus-wrapper"><notextile><figure class="code"><figcaption><span></span></figcaption><div class="highlight"><pre><code class="python3"><span></span><span class="n">text</span> <span class="o">=</span> <span class="n">unicodedata</span><span class="o">.</span><span class="n">normalize</span><span class="p">(</span><span class="s2">&quot;NFKC&quot;</span><span class="p">,</span> <span class="n">text</span><span class="p">)</span>
</code></pre></div></figure></notextile></div>

<p>There are multiple normalization forms, and I chose <code>NFKC</code>: normalize equivalent characters to their canonical one, and
compose all combining characters such that the non-combined and the combined versions become the same.</p>

<p>After fixing the code, I needed to fix the database to fix already-parsed words. In <a href="https://stackoverflow.com/a/19789605/735926">PostgreSQL</a>, we can use
<code>SIMILAR TO</code> to search for strings matching a certain characters range: in my case, the “Combining Diacritical Marks”,
aka <code>0300-036F</code>:</p>

<div class="bogus-wrapper"><notextile><figure class="code"><figcaption><span></span></figcaption><div class="highlight"><pre><code class="sql"><span></span><span class="k">SELECT</span> <span class="n">word</span> <span class="k">FROM</span> <span class="n">word</span> <span class="k">WHERE</span> <span class="n">word</span> <span class="k">SIMILAR</span> <span class="k">TO</span> <span class="s1">&#39;%[\u0300-\u036F]%&#39;</span> <span class="p">;</span>
</code></pre></div></figure></notextile></div>

<p>I use <a href="http://docs.peewee-orm.com/en/latest/">Peewee</a> to interact with Postgres from Python. While it doesn’t support <code>SIMILAR TO</code> out of the box, it’s
simple to use a custom expression:</p>

<div class="bogus-wrapper"><notextile><figure class="code"><figcaption><span></span></figcaption><div class="highlight"><pre><code class="python3"><span></span><span class="n">Word</span><span class="o">.</span><span class="n">select</span><span class="p">()</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">Expression</span><span class="p">(</span><span class="n">Word</span><span class="o">.</span><span class="n">word</span><span class="p">,</span> <span class="s2">&quot;SIMILAR TO&quot;</span><span class="p">,</span> <span class="s2">&quot;%[</span><span class="se">\\</span><span class="s2">u0300-</span><span class="se">\\</span><span class="s2">u036F]%&quot;</span><span class="p">))</span>
</code></pre></div></figure></notextile></div>

<p>Unfortunately this feature is not available in SQLite, the database I use to run my integration tests, and so I had to
adapt the code a little bit: first use <a href="http://docs.peewee-orm.com/en/latest/peewee/sqlite_ext.html#SqliteExtDatabase"><code>SqliteExtDatabase</code></a> instead of <code>SqliteDatabase</code> to get support for
<code>REGEXP</code>, and then use the <code>.regexp</code> operator:</p>

<div class="bogus-wrapper"><notextile><figure class="code"><figcaption><span></span></figcaption><div class="highlight"><pre><code class="python3"><span></span><span class="c1"># SQLite</span>
<span class="n">Word</span><span class="o">.</span><span class="n">select</span><span class="p">()</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">Word</span><span class="o">.</span><span class="n">word</span><span class="o">.</span><span class="n">regexp</span><span class="p">(</span><span class="s2">&quot;[</span><span class="se">\u0300</span><span class="s2">-</span><span class="se">\u036F</span><span class="s2">]&quot;</span><span class="p">))</span>
</code></pre></div></figure></notextile></div>

<p>I was then able to run a quick function to normalize the ~1k words affected by the issue.</p>

<p>To conclude, it seemed a very weird issue at first but in the end it allowed me to learn a few things about Unicode,
Postgres, and Python’s fantastic standard library.</p>


feed_id: https://bfontaine.net/blog/2023/01/09/per-project-gsutil-service-accounts
title: Per-project gsutil service accounts
uri: Per-project-gsutil-service-accounts
url: https://bfontaine.net/blog/2023/01/09/per-project-gsutil-service-accounts/
content:
<p>When using any library for Google Cloud you can specify a service account with <code>GOOGLE_APPLICATION_CREDENTIALS</code>, but
unfortunately that <a href="https://github.com/GoogleCloudPlatform/gsutil/issues/447">doesn’t work</a> when using <code>gsutil</code> in shell scripts. The documentation suggests to use
<code>gcloud auth activate-service-account</code>, but that “activates” the service account for all <code>gsutil</code> invocations, and
doesn’t work if you installed a standalone version of <code>gsutil</code> —without <code>gcloud</code>.</p>

<p>I wanted to have one service account per project so that each project has access to the relevant resources only. The
solution I found is to use a Boto file: this is a ini-like file format used for AWS configuration, but <code>gsutil</code> also
supports it. You can tell <code>gsutil</code> to find such file with <code>BOTO_CONFIG</code> or give it a list of paths to look in with
<code>BOTO_PATH</code>.</p>

<p>In a simple project where the main code is a shell script, the setup would look like this:</p>

<div class="bogus-wrapper"><notextile><figure class="code"><div class="highlight"><pre><code class="text"><span></span>$ ls -a
.boto
script.sh
service-account.json
</code></pre></div></figure></notextile></div>

<p>In <code>.boto</code>:</p>
<div class="bogus-wrapper"><notextile><figure class="code"><figcaption><span></span></figcaption><div class="highlight"><pre><code class="ini"><span></span><span class="k">[Credentials]</span>
<span class="na">gs_service_key_file</span><span class="o">=</span><span class="s">/app/service-account.json</span>
</code></pre></div></figure></notextile></div>

<p>In <code>script.sh</code>:</p>
<div class="bogus-wrapper"><notextile><figure class="code"><figcaption><span></span></figcaption><div class="highlight"><pre><code class="bash"><span></span><span class="ch">#!/bin/env bash -e</span>
<span class="nb">export</span> <span class="nv">BOTO_CONFIG</span><span class="o">=</span>/app/.boto
gsutil ...
</code></pre></div></figure></notextile></div>

<p>This is a bit cumbersome compared to <code>GOOGLE_APPLICATION_CREDENTIALS</code> but it works well.</p>


feed_id: https://bfontaine.net/blog/2023/09/09/another-blog
title: Another blog
uri: Another-blog
url: https://bfontaine.net/blog/2023/09/09/another-blog/
content:
<p>In 2023 I wrote only one article on this blog, but a few more on the <a href="https://tech.bixoto.com/">tech blog</a> of the
company I currently work for, Bixoto:</p>
<ul>
  <li><a href="https://tech.bixoto.com/2023/03/20/deploy-mattermost-on-dokku.html">Deploy Mattermost on Dokku</a></li>
  <li><a href="https://tech.bixoto.com/2023/04/26/add-lets-encrypt-ssl-certificates-on-private-domains-with-dokku.html">Add Let’s Encrypt SSL Certificates on Private Domains with Dokku</a></li>
  <li><a href="https://tech.bixoto.com/2023/07/10/always-encode-your-requests-payloads-in-python.html">Always encode your Requests payloads in Python</a></li>
  <li><a href="https://tech.bixoto.com/2023/08/28/install-avifenc-on-ubuntu-20-04.html">Install avifenc on Ubuntu 20.04</a></li>
  <li><a href="https://tech.bixoto.com/2023/09/04/how-does-bixoto.com-work.html">How does Bixoto.com work?</a></li>
</ul>

<p>This explains why the present blog might look abandoned. It’s not, it’s just that over the years I’ve reduced the time I
spend on side projects and now 95% of my time doing technical stuff on a computer is during work hours.</p>


