Reading short list
(Because I’ll find a book or article series that is important and then life will get in the way)
Sketch algorithms – (Or streaming algorithms) are probabilistic algorithms that can provide answers to certain questions about extremely large sets of data in a short time with known error characteristics. To compute such answers exactly would require a prohibitively large about of time and memory.
Deep learning – Relatively new techniques for training neural networks that have many layers/long credit assignment paths.
Graphical models: Decently paced lecture notes on Bayesian networks.
The long list
- IPAM video lectures
- Tutorial on Bayesian programming: An interesting experiment for an interactive book. The book is a set of iPython notebooks, so you can run the code as you read. The source is on github, so you can modify it as you go along.
- Random projection. Johnson-Lindenstrauss dimension reduction: If you project data to a lower dimension using a random projection matrix the pair-wise distances will largely be conserved if the new dimensionality is of the order of the log of the number of points.
- SCIP lectures
- Common LISP: a gentle introduction to symbolic computation – an easy to read text, mainly on the concepts behind LISP
- The algorithm design manual – a meaty text that is fun to read because it poses interesting problems and then goes into different ways of solving them – the problems are tough and it is fun to see how solutions to them, starting simply, can be approached.
- Jeff Erickson’s algorithm lecture notes
C++/low level programming details
- Undefined behavior in C/C++ – John Reghr   
- A survey of CPU caches (full pdf paper) – Cache lines, size, features, includes testing code.
- Cache Oblivious algorithms – “Cache-oblivious algorithms work by recursively dividing a problem’s dataset into smaller parts and then doing as much computations of each part as possible. Eventually subproblem dataset fits into cache, and we can do significant amount of computations on it without accessing memory”
- A guide to OpenMP – “OpenMP consists of a set of compiler #pragmas that control how the program works. The pragmas are designed so that even if the compiler does not support them, the program will still yield correct behavior, but without any parallelism.”
- Online C++ disassembler – write your code in the left hand pane. The assembly code appears in the right hand side. Magic!
- Xeus/Xeus-cling – C++ Jupyter notebook
Sites I find interesting
- Euclidea: Fun interactive geometry puzzles
- https://lobste.rs/ : The place where I go to for interesting articles on programming and tech.
- Bill Hammack (The Engineer Guy) creates informative and humorous short video demonstrations on EVERYTHING.
- Ask a mathematician/Ask a physicist
- I can’t tell you what Hackery is about, but it’s good.
- A real mathematician and programmer
- Lost in recursion
- A good old fashioned hardware hacker.
- Purely in-browser games
- Red Blob Games – learn computer science concepts through games, I mean, interactive demos.
- The online encyclopedia of integer sequences.
Articles/items that I found fascinating
1500 Archers on a 28.8: Network Programming in Age of Empires and Beyond – An engaging writeup on the design on a multiplayer RTS game in ye olde times of bottle necked machines and networks.
- A simple proof of Gödel’s First Incompleteness Theorem
- Lessons from Optics, The Other Deep Learning: An interesting analogy between machine learning practice and optic lens design.
- C with ABC: From Tom Murphy: A compiler for C89 that compiles to printable ASCII files without generating self-modifying code
- Multi-dimensional analog literals in C++: I don’t actually know what this is doing, but it is an example of template meta-programming in C++.
- The mandelwat set – An extremely engaging, step-by-step tutorial on HTML canvas drawing and animation (lobste.rs)
- An introduction of microarray data analysis – Has a very compact and pragmatic introduction to microarrays. If you are getting started with microarray data analysis, this is a pretty good reference.
- NCBI Geo – In the context of microarrays – a lot of publications upload their data to this public repository. You can search for the data by accession number or by publication/author name, etc, just as you would a paper.
- Modern Garbage Collection – a nice primer on modern GC. (lobste.rs)
- Spikey spheres – and interesting take on the curse of dimensionality and properties of higher dimensional spaces (lobste.rs)
- A nice writeup (from ye olde times) about how to handle lag in gaming over a network. The author worked on the game Descent, which was a pioneering game because it was truly 3D and I spent many an hour in the lab playing this with buddies over a primitive LAN. (lobste.rs)
- Modular arithmetic – a nice note including modular arithmetic, Barrett reduction and division (2WF15 – Discrete Mathematics 2 – Part 1 Algorithmic Number Theory by Benne de Weger)
- Who Needs GPS? The Forgotten Story of Etak’s Amazing 1985 Car Navigation System.
- Four column ASCII
- Historical tax rates in real (2013) dollars
- P != NP – A nice writeup by Scott Aaronson
- Machine Learning best practices – from Martin Zinkevich of Google
- “A unified theory of garbage collection”, Bacon, Chen, Rajan (pdf) an easy to read paper with useful schematics of automatic memory management. Via this thread on lobsters.
- Reservoir sampling – an algorithm to pick a single random sample from a stream of unknown length
- A very engaging article on the discovery and re-discovery of the cause of Scurvy.
- A nice tutorial on Kalman filters.
- A simple generative function for Fibonacci numbers.
(4 << n*(3+n)) // ((4 << 2*n) - (2 << n) - 1) & ((2 << n) - 1)
- “Graphical algebra” interesting algebraic/trigonometric concepts lovingly described using animations of graphs. From the author of acko.net
- A painstakingly well done, interactive, visual introduction to machine learning.
- A extraordinary exposition of visualizing dimensionality reduction.
- Analysis of a raytracer written to fit on the back of a business card (almost).
- A nice article on camera shutters.
- Prime number spirals.
- A bezier clock.
- The xkcd pixels comic.
- A visual exposition of markov chains.
- Add up the sine waves. Fun interactive fourier explorer.
- An entertaining account by Tom Mahood, of his work with James Woodward, testing an inertial drive.
- A very good article on how malloc and co work to allocate/deallocate memory
- “We can’t send email more than 520 miles” – an epic debugging tale
- Repairing an underground high voltage power line
- A “magic carpet” ride to a suspended high voltage line
- Angels-on-a-Pin, AKA a tale of academic sabotage in the best of traditions
- The statistical analysis of fMRI data. Martin A. Lindquist, Statist. Sci. Volume 23, Number 4 (2008), 439-464 – A thorough but easy to read introduction to fMRI. Though the focus is on statistical methods, it has a great introduction to fMRI and some important tradeoffs that are characteristic of fMRI studies.
- Fifty psychological and psychiatric terms to avoid. I agree with most of them, though a few are acceptable simplifications for non-specialist writing
- The black box that pumps out primes. (And a related paper about a simple prime number generating formula)