In this paper we examine how human-machine interaction in the
legal sector is suggested to be regulated in the EU’s recently
proposed Artificial Intelligence Act. First, we provide a brief
background and overview of the proposal. Then we turn towards the assessment of high-risk AI systems for the legal tasks as well as the obligations for such AI systems in terms of human-machine interaction. We argue that whereas the proposed definition of AI system is broad, the concrete high-risk area of ‘administration of justice and democratic processes’, despite coming with considerable legal uncertainty, is narrow and unlikely to extent into many uses of legal AI and IA systems. Nonetheless, these regulatory developments may be of great relevance for current and future legal AI and IA systems.