Bias, memorisation, carbon cost, copyright, the EU AI Act, open weights, AGI claims
|
Next Words - Lesson 17 - Ethics, Society, the Future (pdf)
|
271.31 K |
GPT-3's training run is estimated at about 500 tonnes of CO2 on an average US grid and about 25 tonnes on Norwegian hydropower, a factor of 20. Bias is traced through four stages, data, annotation, amplification and deployment, and a 90/10 corpus sampled at T = 0.7 gives the majority continuation 95.8% of the time. A per-token loss of 0.010 nats gives a probability of 0.607 for 50 verbatim tokens, and the defences are deduplication, DP-SGD, output filters and unlearning. Inference over a year, at ten times the training figure, is 5,000 tonnes against 500 for training. The law is treated through fair use, the four risk tiers of the EU AI Act and open weights. Students price GPT-3's training run across four grid intensities, swap names in four prompts and score sixteen StereoSet-style triples.