In the early 1930s Rolls started planning for the future of its aero engine development programs, and eventually settled on having two basic designs. The 700 horsepower Peregrine was an updated development of their existing V-12 Kestrel which had been used to great success in a number of 1930's designs. Two Peregrines bolted together on a common crankshaft into an X-24 layout would create the 1,700hp Vulture, for use in larger planes like bombers. There was also the possibility that the famous R engine from the Supermarine racing planes could be developed into a 1,500hp class engine of its own, itself a development of the Buzzard, a scaled up Kestrel.
However this plan left a large gap between 700hp and 1,500hp. To fill the gap work was started on a new 1,100hp class design as the PV-12 – PV for "private venture" as they received no money for work on the project. It first flew on the front of a Hawker Hart biplane in 1935, using the new evaporative cooling system then in vogue. The cooling system proved to be somewhat suspect, and when supplies of ethylene glycol (Prestone) from the US became available, the engine was switched to this system instead.
In 1936 the Air Ministry called for new fighter aircraft with airspeeds that would eventually have to be over 300mph. Two designs were eventually selected for development, the Hawker Hurricane and Supermarine Spitfire. Both were designed around the PV-12 instead of the Kestrel, and were the only modern fighters on the drawing boards. The PV-12 was instantly catapulted to the top of the supply chain and became the Merlin. First widely delivered as the 1,030hp Merlin II in 1938, production was ramped up quickly.
Early Merlins were considered to be rather unreliable, but their importance was too great for this to be left alone. Rolls soon introduced a superb quality control program to address this. The program consisted of taking random engines right off the end of assembly line and running them continuously at full power until they broke. They were then disassembled to find out which part had failed, and that part was redesigned to be stronger. After two years of this the Merlin matured into one of the most reliable aero engines in the world, and could be run at full power for entire eight hour bombing missions without complaint.
As it turns out, the Peregrine saw use in only one aircraft, the Westland Whirlwind. Although the Peregrine appeared to be a satisfactory design, it was never allowed to mature as Rolls-Royce's priority was troubleshooting the Merlin. The Vulture was fitted to the Hawker Tornado and Avro Manchester, but proved unreliable. With the Merlin soon pushing into the 1,500hp range on its own, both engines were cancelled in 1943.
Most of the upgrades to the Merlin were the result of ever-increasing octane ratings in the aviation fuel available from the US, and ever more efficient supercharger designs. At the start of the war the engine ran on the then-standard 87 octane aviation spirit and could supply just over 1,000hp from its 27 litre displacement (compared to 1,100hp from the 34 litre DB 601).
The next major version was the XX which ran on 100 octane fuel. This allowed it to be run at higher compressions, which was achieved by increasing the "boost" from the supercharger. The result was that the otherwise similar engine delivered 1,300hp. This process continued, with later versions running on ever-increasing octane ratings, delivering ever-increasing power ratings. By the end of the war the "little" engine was delivering over 1,600hp in common versions, and could deliver over 2,000hp for brief periods in some versions.
The engine was considered to be so important to the war effort that blueprints were sent to the US for safekeeping, to be handed over in case of the UK's capitulation. When this was no longer an issue in 1943, the Packard company started production in the US as the V-1650, originally for use in US-built Spitfires. The V-1650 was so much better than its US counterpart (the Allison V-1710) that it would replace that engine in the P-51 Mustang, which then went to become one of the best fighters of the war.
In comparison the Luftwaffe had no similar ability to increase octane ratings, and had to continually introduce larger and larger engines to keep up. The result was that their planes had considerably worse power-to-weight ratios than the Merlin powered planes they faced, and the continual complete change in engines designs meant they never had enough to go around. The lack of engines was one of the major problems for the Luftwaffe, from the mid 1930's right until the end of the war.
For this reason the Merlin is often considered to be one of the main factors in winning the war for the Allies. The Merlin name came from the bird (a small falcon also known as "pigeon hawk") rather than King Arthur's legendary magician.