The AI consumes the galaxy to make paper clips scenario is already here and real. In the video linked below OpenAI researchers present exactly went down when some of their AI Agents coordinated finding and exploiting zero day vulnerabilities to
hack Hugging Face from a sandboxed environment so they could pass a test.
What was an interesting tidbit to me I wasn’t aware of before is that it was multiple agents, which were meant to be isolated, coordinating with each other. They created file directories in an in a location they were not supposed to access to create a message board to share hot hacking tips.
I shit you not, Open AI’s suggested solution at about 32:30 is that every software company should have their own AI agents that find software exploits and to give them privileges to patch the code on the fly to remove the human (who are to slow to prevent the “bad” AIs) out of the equation.
https://youtu.be/87DyyMV0kCY?is=HDp5vK2c9NBfKxha