He would suffer his first defeat in his third fight, when faced against Doug McClendon, who beat him by a decision in six on January 26, 1971 in New York. He won five bouts in a row, and then lost by decision in eight to future Roberto Duran world title challenger Edwin Viruet.
He began 1972 by losing to another future world title challenger, Diego Alcala, by knockout in round eight, but he won his three other fights that year.
In 1973, he began, once again, by losing to another future world title challenger, Miguel Montilla, by a decision in ten. Before the year was over, however, he was able to avenge that defeat, defeating Montilla by a knockout in round eight, and he won seven of his eight other fights that year.
By 1974, he began climbing up the Jr. Lightweight rankings, going 8-2 that year. He beat his own future world title challenger Sigfredo Rodriguez by a knockout in round one, and former world champion Ricardo Arredondo by a disqualification in round eight.
In what could be considered ironic, Escalera received his first world championship try after drawing in ten rounds with Francisco Villegas: On July 4, 1975, he fought the WBC world Jr. Lightweight champion Kuniaki Shibata in Mito, knocking Shibata out in round two to become a world Jr. Lightweight champion.
He defended his title 10 times, including wins over Ray Lunny, Sigfredo Rodriguez and Tyrone Everrett, before losing it on January 28 of 1978 to Alexis Arguello of Nicaragua by a knockout in round 13 in Bayamon. In this fight, Escalera suffered a broken nose and tooth, a cut on his tongue and above his left eye, and a closed right eye before submitting to Arguello.
Escalera and Arguello had a rematch, on February 4, 1979 in Rimini, and Escalera held a small lead on the judges' cards after 12 rounds, but he was knocked out, once again, in round 13 by Arguello. Shortly after drawing with Antonio Cruz in ten rounds in October of that year, he announced his retirement.
In 1980, Escalera dedicated himself to the sport of wrestling, competing on the Puerto Rican professional wrestling circuit, but in 1981, he launched a boxing comeback. He lost to future world Jr. Welterweight champion Gene Hatcher in San Antonio, but he beat former world title challenger Maurice Termite Watkins at The Battle of the Champions' undercard in Miami, to complete his 1982 boxing campaign.
In 1983, he avenged his loss to Hatcher, dropping him in round six and scoring a 10 round unanimous decision win at the Duran-Davey Moore world title fight's undercard in the Madison Square Garden, but on September 15 of that year, he lost to future world title challenger Charlie White Lighting Brown at the Ray Mancini-Orlando Romero world title bout's undercard.
After that fight, Escalera announced his retirement from boxing once again, and the second time around, he was able to stay retired. Shortly after, it was discovered that he had eyesight problems and had been fighting almost blinded for his last few fights.
Escalera and Samuel Serrano became the first two Puerto Ricans to be world champions at the same division during the same time. They would be emulated years later by Ossie Ocasio and Carlos De Leon in the Cruiserweight division. There was frequent talks about facing Escalera and Serrano in a unification world title bout. The bout, however, never materialized.
Escalera became a house-hold name in Puerto Rico during his tenure as world champion. He appeared on tv commercials and was a popular public figure.
The avid autograph signer left a boxing record of 49 wins, 14 losses and 2 draws, with 29 wins by knockout. His two fights with Arguello are considered by many to be boxing classics, Arguello-Escalera I making The Ring magazine's list of 100 greatest fights of all time at number 31, and Arguello-Escalera II making it at number 68.
Escalera now enjoys the fruits of his career in his farm in Puerto Rico. His son, Alfredo Escalera Jr, is currently an amateur boxer who is based in Florida.