I’m sure somebody has made these kinds of analyses in much greater detail, but I wanted to get a sense of the computational limits that we are presently at. Caution: Many of the following numbers are pulled out of the internet without serious effort.

Energy efficiency

At its developed peak, the brain might have 200 trillion synapses and consume 1760 kJ/day. Making this a ratio, we get 200 trillion synapses / 20W = 10 trillion synapses / watt. Suppose 1 synapse performs at least 10 ‘operations’ per second. Then the brain performs at least 100 TFLOPS with an ideal efficiency exceeding 100 TFLOPS/watt. Compare this to machine computation:

  • This paper (fig 5) says that in 2020, GPUs reach 100 GFLOPS/watt. However it notes that energy efficiency is exponentially increasing (rough estimate, 10x every 10 years)
  • The v3-32 TPU Pod delivers ~1680 TFLOPS (see below) with estimated power consumption 200W/core*32cores = 6.4kW. This makes about 250GFLOPS/watt. Only about 3 orders of magnitude less than this brain estimate.

Compute cost efficiency

Suppose it costs $10/day to sustain a human brain. Using the above measures, then the brain can perform at least 2000 trillion operations per second using only $0.0001… for a single second. This makes 17280000 TFLOP/$ or 17 exaflops per dollar. The v3-32 TPU Pod ideally approaches 4 times the single 8 core TPU performance = 4 * 420TFLOPS = 1680TFLOPS at a price $10,512 / month or $0.00400219298 per second. This makes 1680TFLOP/$0.00400219298 or 419769 operations per dollar or about 0.5MFLOP/$. Digital compute infrastructure looks on the order of 10^18 times less cost-efficient than the brain’s biological computation.

Raw Compute

The 100 TFLOPS brain estimate looks rather small. This number is easily surpassed by a single TPU pod in terms of brute computational speed – but definitely not efficiency. Still, the TPU pod is not unaffordably expensive. Maybe this infrastructure is sufficient for developing human-level AI!

Update (7/14/23): I think it can be trained in under $1B with novel architectures and creative techniques.