Fundé Clear : AI and the Joys of Learning from First Principles
How and why our book, The Little Learner: A Straight Line to Deep Learning came to be.
If you went through higher education in India, you have surely heard the phrase iske fundé clear hain (“Their fundé are clear”) or perhaps more likely said of yourself yaar mere fundé gol hain (“Man, my fundé are messed up”). If you didn’t go through higher education in India, I’m sure you’re wondering why all this fuss about fundé. What does it even mean?
Nominally, the word fundé (फ़ंडे, pronounced “fun-day”, like Sunday) is the Hindi plural form of funda (फ़ंडा), which itself is a colloquial contraction of the English word fundamental. Thus fundé means fundamentals. In an educational context, it refers to the strength of one’s understanding of the fundamentals of a given subject. Those who have their fundé clear in a subject are bound to do well in the dreaded end-of-semester/year exams.
The idea acknowledges that all subjects have an axiomatic basis that, if mastered, allows one to build a thorough understanding of the subject. The alternative to an axiomatic understanding is rote memorization. Note that either strategy works for getting through exams, but those that have their fundé clear occupy a special place of reverence amongst their peers. Similarly the lament of having your fundé messed up is the expression of fear that you’re not likely to do well in the subject.
For most people going through higher education in India, discovering the axiomatic foundations of a subject is a torturous process. The pedagogy is too often convoluted unless one is blessed with a gifted teacher. Those who do eventually get their fundé clear mostly do so with some struggle and hard work.
Fundé are a part of an underlying system of values
Culturally, it goes even beyond exams. It pays homage to the axiomatic principles of life itself. It’s a common question in many daily conversations: iska funda kya hai? (“What’s the funda behind this?”) where the “this” could refer to something as mundane as a toaster knob or as profound as the existential questions in life.
It is not unusual to appreciate a peer for having a clear sense of prioritization in their life: iske life ke fundé clear hain (Their fundé of life are clear). There is a reverence for the general idea that there are fundamental axioms in all aspects of life. And further, mastering those fundamentals generally leads to better things in life and is a desirable state of mind.
This universality of fundé in life has always been a huge influence on me. If one can derive things from first principles (i.e., the fundé are clear), then the retention of worthless knowledge becomes unnecessary, and life become much simpler.
Which brings me to artificial intelligence, and more specifically deep learning.
The pedagogy of deep learning is forbidding
I first became aware of the rapid progress occurring in deep learning around 2012. A subsequent talk at Indiana University by the then Chief Scientist of Microsoft, Peter Lee, brought my attention to more new developments. Very quickly, I began scouring the literature on the subject, reading anything and everything I could find on it.
What I discovered was that most literature was grounded in anything but first principles. One was (and still mostly is) routinely expected to be proficient in vector calculus, linear algebra, and probability and statistics. If your fundé in these subjects are not clear, you are fundamentally messed up, because your fundé in deep learning will remain similarly unclear.
On top of that, the pedagogy of the subject reinforces this mindset — almost every book on the subject begins with “mathematical preliminaries” that are particularly deceptive. They are condensed and consequently much denser than other better written books in the subjects, so just getting past them is an exercise.
In other words, the pedagogy of deep learning tends to intimidate. It keeps those who have not embraced the M in STEM away, and this includes most developers in today’s tech industry.
Things have definitely improved since 2012 and there are newer resources that have tried to bridge this gap, but they often choose to black-box the mathematics into a code library, rather than bring the mathematical intuition forward and build it from first principles to show causal links between the different ideas in the subject.
Enter The Little Learner
That brings me to my co-author Daniel P. Friedman, who has been teaching computer science at Indiana University for the last half-century. He is famous for his Socratic approach to teaching very complex ideas in computer science entirely from first principles. His other books: The Little Lisper, The Little Schemer, The Seasoned Schemer, The Reasoned Schemer, The Little Prover, The Little Typer, and more have laid the groundwork of this form of pedagogy. Full disclosure: he is also my Ph. D. advisor from way-back-when.
If there was anything that could be done to provide a better pedagogical path to deep learning, it would have to involve him! We ran into each other on Friday, the 13th of April, 2018, at the over-crowded, official opening of the Luddy School of Informatics, Computing, and Engineering and we decided to write a book on machine learning based on this very deep conversation directly following the close of the event.
Anurag: I want to write a little book with you.
Dan: Let’s do it!
…
a few seconds later
…
Dan: What’s the topic?
Anurag: Machine learning
Dan: Now, that will be a worthy challenge!
And the rest of the time we reminisced ...
We have since then worked together these last four-plus years to write the latest in the series of “Little” books: The Little Learner: A Straight Line to Deep Learning. It develops the complex ideas in deep learning from first principles without appealing to anything more than basic high school algebra and geometry, and a knowledge of programming.
In other words, it is designed to get your fundé clear and experience the joy of learning about this form of AI from first principles. We are happy to announce that the book is now generally available and shipping, through the auspices of MIT Press (Cambridge, MA).
We hope this little foray into deep learning will be fun for you, and we hope that it’s as interesting to read as we have found it to write.
Bon appétit, from us — Daniel P. Friedman and Anurag Mendhekar.
