AI Safety

How do we measure, study and reason about the safety and security of many agents interacting, rather than one agent in isolation?

This workshop from the Institute for Decentralized AI presented the latest academic research into this area. The future is going to be populated with imperfect agents that will need to be able to coordinate with untrusted others. As activity starts to depend on long chains of agentic action, we need to ensure that each edge in the network is secure enough that the probability of failure across the entire chain is within acceptable bounds. This is critical if agentic economies of scale are to emerge.

It was fascinating to see how ideas from cryptography are being applied to this problem domain to reason about the limits of possibility. This is currently a very nascent field, where the rules and assumptions underpinning the discipline are still being discovered, defined and challenged. The bottleneck holding back the space is figuring out which of those assumptions to challenge and which to base our systems around.

Other topics included:

• How do we resolve the tension between stochastic AI agents and deterministic smart contracts, and get the best of both worlds — the open-ended intelligence of the one, the bounded guarantees of the other?
• What can we safely compartmentalise, decompose, and delegate down the chain?

Interesting papers I will add to my to read pile:

• Scalable AI Safety via Doubly Efficient Debate
• Multi-Agent Risks from Advanced AI