On 5 October 1954 Evatt gave an aggressive speech against 'disloyal elements' within the Labor Party, which aimed "to deflect the Labor Movement from the pursuit of established Labor objectives and ideals." The speech caused ructions within the ALP leading many to question Evatt's position.
Labor's caucus rejected by 52 to 28 votes a motion for a spill moved by Senators George Cole and James Fraser. Deputy leader Arthur Calwell and Allan Fraser would have stood for election as Leader and deputy leader respectively in the event of a spill occurring. After the ballot, Evatt insisted on counting the names for and against, which only furthered his opponents animosity.