The rise in the use of AI in most key areas of business, from sales to compliance to financial analysis, means that even the highest levels of corporate governance will be impacted, and that corporate leaders are duty-bound to manage both the responsible development and the legal and ethical use of AI. This transformation will directly impact the legal and ethical duties and best practices of those tasked with setting the ‘tone at the top’ and who are accountable for the firm’s success. Directors and officers will have to ask themselves to what extent should, or must, AI tools be used in both strategic business decision-making, as well as monitoring processes. Here we look at a number of issues that we believe are going to arise due to the greater use of generative AI. We consider what top management should be doing to ensure that all such AI tools used by the firm are safe and fit for purpose, especially considering avoidance of potential negative externalities. In the end, due to the challenges of AI use, the human component of top corporate decision-making will be put to the test, to prudentially thread the needle of AI use and to ensure the technology serves corporations and their human stakeholders instead of the other way around.