Agentic artificial intelligence (AI) systems in clinical environments perform multi-step reasoning, tool invocation, and real-time actions over electronic health record (EHR) workflows. While healthcare systems commonly enforce credential- and APIlayer role-based access control (RBAC), agentic architectures introduce an additional risk: semantic role manipulation at inference time, where natural-language interactions can steer an agent's tool-routing decisions toward unintended privileges when role context is inferred rather than cryptographically pinned. This paper formalizes Inference-Time Role Drift Attacks (IRDA), in which an adversary uses multi-turn coercion and paraphrasing to shift an agent's perceived operational role and trigger unauthorized tool requests. We propose a Role-Constrained Agentic Firewall (RCAF) that detects role drift using embedding-based consistency checks over multiple signals, including agent reasoning outputs, conversation context, and user messages, and blocks suspicious tool invocations. Using 100 synthetic patient profiles and 2,800 attack episodes, IRDA achieves a 92% attack success rate against a baseline that authorizes tool access based on inferred roles. RCAF reduces successful unauthorized tool invocations to 0% at conservative thresholds while detecting drift in 92% of adversarial episodes; however, it also blocks legitimate escalation workflows, exposing a critical security-usability trade-off. We report ablations over embedding models (MiniLM vs. MPNet), detection signals, and threshold sweeps. Overall, the results show that credential-layer controls alone are insufficient when role inference drives tool authorization and that semantic-layer monitoring can mitigate IRDA but requires workflow-aware escalation handling.