Carrillo is the son of the prominent Socialist leader Wenceslao Carrillo and was already as a 13-year old a member of the Spanish Socialist Workers' Party (PSOE). He was 18 when he became General Secretary of a bloque of the Workers Party of Marxist Unification (POUM) - a left-wing of the Socialist Party.
He collected both the Young Socialist and Young Communist in a common union in 1934, and became a member of the Communist Party of Spain in 1936. He lead the communist forces in Madrid during the Spanish Civil War. After the Spanish Civil War he moved to Paris, and tried to organize the party while being in exile there.
He became the General Secretary of the Communist Party of Spain in 1960, while Dolores Ibarruri was president. Carrillo enforced the party's position among the working class and intellectual groups. He returned secretly to Spain in 1976, and was arrested shortly after but ultimately released within days.
Carrillo was elected as a deputy of the Cortes (Spanish legaslative body) in 1977, and re-elected again in 1979 and 1982. After a serious crisis within his party (PCE) in November 1983, Carrillo had to leave his post as the party leader.
In 1986 Carrillo formed a new party, The Spanish Labor Party, but it was later dissolved in 1991.