GoKawiilmicrocrad
microcrad is a tiny scalar-valued automatic differentiation engine for C, with a small neural network implementation built on top of it. It is a re-implementation of Andrej Karpathy's micrograd in C, written for people who want to understand how backpropagation really works.
Like the Python original, microcrad operates on scalars, not tensors. Every number that takes part in a computation is a node in a graph, every operation records how it was produced, and a single backward pass walks the graph in reverse to compute the derivative of the output with respect to every input. There is no vectorization, no GPU, no clever tricks: just the chain rule applied one scalar at a time. On top of this engine sits a multi-layer perceptron, so you can build a network, run a forward pass, call backward, and do gradient descent, all in C.
This repository is first and foremost an educational implementation. It is meant to be read, experimented with, and tested. It is not a production autograd package, not a practical deep-learning framework, and not optimized for large datasets or numerical robustness.
The whole thing is built around two ideas:
A Value , which is a single node in the computation graph.
, which is a single node in the computation graph. Reference counting, which is how microcrad knows when a Value is no longer part of any graph and can be freed.
Almost everything in the documentation below is a consequence of these two ideas, so it is worth keeping them in mind.
How Values work
The fundamental type is Value . A Value wraps a single double and, when it is the result of an operation, remembers the operands it was computed from:
... continue reading