Jones learned his keyboard skills from his father, Joe Baldwin, who was a pianist and arranger for big bands in the 40's and 50's, notably with the Ambrose Orchestra. His mother was also in the music business which allowed the family to often perform together touring around England. His influences ranged from the blues of Big Bill Broonzy, the jazz of Charles Mingus, to the classical piano of Rachmaninov. Jones was a student at Christ College boarding school in Kent where he formally studied music. At the age of 14, he became choirmaster and organist at a local church and during that year, he also bought his first bass guitar, a Dallas solid body electric followed by a Fender Jazz bass. The fluid playing of Chicago musician Phil Upchurch made him decide to take up the bass guitar. Jones continued to use that Fender bass up until 1975.
Jones joined his first band at 15, which was called The Deltas. He then played bass for jazz-rock London group, Jett Blacks. His big break came in 1962 when he met Jet Harris & Tony Meehan and played bass for their band for two years, during which time they scored a Number 1 hit with "Diamonds". In 1964, Jones began session work with Decca Records on the recommendations of Tony Meehan. Between 1964 and 1968 he was much in demand arranging, playing keyboards or bass guitar for artists including The Rolling Stones, Herman's Hermits, Donovan, Jeff Beck, Cat Stevens, Rod Stewart, Shirley Bassey, Lulu, and numerous others. As well as recording sessions with Dusty Springfield, Jones also played bass for her Talk Of The Town series of performances. His arranging and playing on Donovan's "Sunshine Superman", resulted in producer Mickie Most using his services as choice arranger for many of his own projects, with Tom Jones, Nico, Wayne Fontana, The Walker Brothers, and many others.
It was during the sessions for Donovan's Hurdy Gurdy Man that Jones met Jimmy Page and the two would eventually link up again during the making of the Yardbirds Little Games album, with Jones contributing orchestral arrangements and playing cello on the opening track "Little Games". Although the album did not fare well commercially, the session did result in Jones and Page discussing to work again in the future. A year later, as Chris Dreja had decided to leave Page’s group and take up photography, Jones was Page's first choice for bass player in The New Yardbirds, later to become Led Zeppelin. His decision to leave session work and join a group was due to his desire to express his artistic creativity. Despite the spotlight being placed on Robert Plant and Jimmy Page, Jones' temperament, musicianship, and experience were crucial to creating the bedrock sound and success of Led Zeppelin.
Jones was responsible for the classic bass lines of the group, notably the descending notes on "Dazed and Confused" (Led Zeppelin I), the funk influenced riffs of "The Lemon Song" and "What Is and What Should Never Be" (Led Zeppelin II), and the power crunch of "Black Dog" (Four Symbols). After "retiring" his Fender Jazz bass in 1975, Jones switched to using custom designed Alembics. His keyboard skills added an eclectic dimension that realised Led Zeppelin as more than just a heavy metal band, most notably on the delicate "The Rain Song" (Houses of the Holy) played on a Mellotron, and the eastern scales of "Kashmir" (Physical Graffiti). On live performances, Jones’ keyboard showpiece was "No Quarter", often lasting for up to half-an-hour and included snatches of "Amazing Grace" and variations of classical pieces by composers such as Rachmaninov. His diversity for the group also extended to other instruments, which included unusual double and even triple necked mandolins.
Jones involvement with Led Zeppelin however did not halt his session work. In 1969 he returned to the studio to play bass guitar on Family Dogg's Way of Life album, in 1970, keyboards for guitarist Peter Green on his solo album The End of the Game. Jones was Madeline Bell's first choice to produce and arrange her 1974 album Comin' Atcha. He has also played keyboards on many Roy Harper albums and contributed to Wings Back to the Egg Rockestra.
Since 1980 his collaborators have included REM, Heart, Ben E. King, The Mission, La Fura Dels Baus, Brian Eno, and The Butthole Surfers. He appeared on a number of sessions and videos for Paul McCartney and was involved in the soundtrack of the film Give My Regards to Broad Street. In 1986, Jones was asked by director Michael Winner to provide the soundtrack for the film, Scream for Help, with Jimmy Page appearing on two tracks. He recorded and toured with Diamanda Galás on her 1994 album, The Sporting Life (co-credited to John Paul Jones). Jones set up his own recording studio called Sunday School as well being involved in his daughter’s (Jacinda Jones) singing career.
Zooma, his debut solo album, was released in September 1999 and followed up in 2001 by Thunderthief which includes his debut as a solo vocalist.
John Paul Jones’s first solo recording was a single for Pye Records in April 1964 which featured "A Foggy Day in Vietnam". The B-side was "Baja".