Reading short list
(Because I’ll find a book or article series that is important and then life will get in the way)
Sketch algorithms – (Or streaming algorithms) are probabilistic algorithms that can provide answers to certain questions about extremely large sets of data in a short time with known error characteristics. To compute such answers exactly would require a prohibitively large about of time and memory.
Deep learning – Relatively new techniques for training neural networks that have many layers/long credit assignment paths.
Graphical models: Decently paced lecture notes on Bayesian networks.
The long list
- IPAM video lectures
- Tutorial on Bayesian programming: An interesting experiment for an interactive book. The book is a set of iPython notebooks, so you can run the code as you read. The source is on github, so you can modify it as you go along.
- Random projection. Johnson-Lindenstrauss dimension reduction: If you project data to a lower dimension using a random projection matrix, the pair-wise distances will largely be conserved if the new dimensionality is of the order of the log of the number of points.
- SCIP lectures
- Common LISP: a gentle introduction to symbolic computation – an easy to read text, mainly on the concepts behind LISP
- The algorithm design manual – a meaty text that is fun to read because it poses interesting problems and then goes into different ways of solving them – the problems are tough and it is fun to see how solutions to them, starting simply, can be approached.
- Jeff Erickson’s algorithm lecture notes
C++/low level programming details
- Undefined behavior in C/C++ – John Reghr   
- A survey of CPU caches (full pdf paper) – Cache lines, size, features, includes testing code.
- Cache Oblivious algorithms – “Cache-oblivious algorithms work by recursively dividing a problem’s dataset into smaller parts and then doing as much computations of each part as possible. Eventually subproblem dataset fits into cache, and we can do significant amount of computations on it without accessing memory”
- A guide to OpenMP – “OpenMP consists of a set of compiler #pragmas that control how the program works. The pragmas are designed so that even if the compiler does not support them, the program will still yield correct behavior, but without any parallelism.”
- Online C++ disassembler – write your code in the left hand pane. The assembly code appears in the right hand side. Magic!
- Xeus/Xeus-cling – C++ Jupyter notebook
Sites I find interesting
- Euclidea: Fun interactive geometry puzzles
- https://lobste.rs/ : The place where I go to for interesting articles on programming and tech.
- Bill Hammack (The Engineer Guy) creates informative and humorous short video demonstrations on EVERYTHING.
- Ask a mathematician/Ask a physicist
- I can’t tell you what Hackery is about, but it’s good.
- A real mathematician and programmer
- Lost in recursion
- A good old fashioned hardware hacker.
- Purely in-browser games
- Red Blob Games – learn computer science concepts through games, I mean, interactive demos.
- The online encyclopedia of integer sequences.
Articles/items that I found fascinating
- Robert Hodgin‘s procedural system for generating meandering rivers on maps – this is a fine piece of computer generated art, enjoyable to those who have a taste for old maps.
- Charles McNair CNN opinion piece on the miracle of vaccines – I found it notable because the author describes how his aunt was stricken by polio, lost use of her diaphragm, was in an iron lung for three months, and then recovered partially.
- An intuitive explanation for how planes fly still eludes us – Scientific American tells us that the Bernoulli principle is a half truth, Einstein did not understand flight, and even experts in computational fluid dynamics can not explain simply how planes fly.
- Floating point visually explained – Fabien Sanglard explains the exponent and mantissa of the IEEE floating point using an intuitive visual exposition.
- Adam Rosenberg‘s site – Just a website of a man from an earlier era.
- TV backlight compensation – Pekka Väänänen’s second hand TV backlight’s red LEDs were failing in a splotchy pattern across the screen. He fixed it in an ingenious manner. It was also fun to learn of the MPC-BE video player which supports live shader editing.
- A scalable pipeline for designing reconfigurable organisms – The paper describes a study where cells are harvested from frog embryos and mechanically manipulated so that they respond in predictable ways, creating made to order … moving things. I can not comment on the rigor of the study or the quality but I’m glad people are getting funded to work on this kind of out there stuff.
- First million integers represented in 2D projection of a much larger prime factor based space. – the first million integers, represented as binary vectors indicating their prime factors, and laid out using the UMAP dimensionality reduction algorithm by Leland Mcinnes. Each integer is represented in a high-dimensional space, and gets squished down to 2D so that numbers with similar prime factorisations are closer together than those with dissimilar factorisations. (johnhw)
- 1500 Archers on a 28.8: Network Programming in Age of Empires and Beyond – An engaging writeup on the design on a multiplayer RTS game in ye olde times of bottle necked machines and networks.
- I can’t use Rift S, and neither can you – Palmer Lucky is the founder of Occulus VR. This was an interesting article because it introduced me to inter pupillary distance as being a key driver of problems with VR.
- A simple proof of Gödel’s First Incompleteness Theorem
- Lessons from Optics, The Other Deep Learning: An interesting analogy between machine learning practice and optic lens design.
- C with ABC: From Tom Murphy: A compiler for C89 that compiles to printable ASCII files without generating self-modifying code
- Multi-dimensional analog literals in C++: I don’t actually know what this is doing, but it is an example of template meta-programming in C++.
- The mandelwat set – An extremely engaging, step-by-step tutorial on HTML canvas drawing and animation (lobste.rs)
- An introduction of microarray data analysis – Has a very compact and pragmatic introduction to microarrays. If you are getting started with microarray data analysis, this is a pretty good reference.
- NCBI Geo – In the context of microarrays – a lot of publications upload their data to this public repository. You can search for the data by accession number or by publication/author name, etc, just as you would a paper.
- Modern Garbage Collection – a nice primer on modern GC. (lobste.rs)
- Spikey spheres – and interesting take on the curse of dimensionality and properties of higher dimensional spaces (lobste.rs)
- A nice writeup (from ye olde times) about how to handle lag in gaming over a network. The author worked on the game Descent, which was a pioneering game because it was truly 3D and I spent many an hour in the lab playing this with buddies over a primitive LAN. (lobste.rs)
- Modular arithmetic – a nice note including modular arithmetic, Barrett reduction and division (2WF15 – Discrete Mathematics 2 – Part 1 Algorithmic Number Theory by Benne de Weger)
- Who Needs GPS? The Forgotten Story of Etak’s Amazing 1985 Car Navigation System.
- Four column ASCII
- Historical tax rates in real (2013) dollars
- P != NP – A nice writeup by Scott Aaronson
- Machine Learning best practices – from Martin Zinkevich of Google
- “A unified theory of garbage collection”, Bacon, Chen, Rajan (pdf) an easy to read paper with useful schematics of automatic memory management. Via this thread on lobsters.
- Reservoir sampling – an algorithm to pick a single random sample from a stream of unknown length
- A very engaging article on the discovery and re-discovery of the cause of Scurvy.
- A nice tutorial on Kalman filters.
- A simple generative function for Fibonacci numbers.
(4 << n*(3+n)) // ((4 << 2*n) - (2 << n) - 1) & ((2 << n) - 1)
- “Graphical algebra” interesting algebraic/trigonometric concepts lovingly described using animations of graphs. From the author of acko.net
- A painstakingly well done, interactive, visual introduction to machine learning.
- A extraordinary exposition of visualizing dimensionality reduction.
- Analysis of a raytracer written to fit on the back of a business card (almost).
- A nice article on camera shutters.
- Prime number spirals.
- A bezier clock.
- The xkcd pixels comic.
- A visual exposition of markov chains.
- Add up the sine waves. Fun interactive fourier explorer.
- An entertaining account by Tom Mahood, of his work with James Woodward, testing an inertial drive.
- A very good article on how malloc and co work to allocate/deallocate memory
- “We can’t send email more than 520 miles” – an epic debugging tale
- Repairing an underground high voltage power line
- A “magic carpet” ride to a suspended high voltage line
- Angels-on-a-Pin, AKA a tale of academic sabotage in the best of traditions
- The statistical analysis of fMRI data. Martin A. Lindquist, Statist. Sci. Volume 23, Number 4 (2008), 439-464 – A thorough but easy to read introduction to fMRI. Though the focus is on statistical methods, it has a great introduction to fMRI and some important tradeoffs that are characteristic of fMRI studies.
- Fifty psychological and psychiatric terms to avoid. I agree with most of them, though a few are acceptable simplifications for non-specialist writing
- The black box that pumps out primes. (And a related paper about a simple prime number generating formula)