Add proper lazy deserialization

[ilevkivskyi]

On Linux with Python 3.12 this gives (compared to 1.20.0):

• almost 2x less memory overhead per parallel worker
• 3-4% faster cold parallel runs
• (most notably) ~2x faster mypy -c 'import torch' with warm cache, on my desktop warm torch now takes 250-300ms

cc @JukkaL

[github-actions]

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅