He was born in Hausneindorf, his father the organ builder Adolf Reubke. He went to the conservatory in Berlin where he studied piano under Theodor Kullak. On the recommendation of Hans von Bülow, he went to Weimar to study with Franz Liszt in 1856 and became one of his favourite pupils. When Reubke died two years later in Pillnitz, Liszt wrote a letter of sympathy to his father, saying "Truly no one could feel more deeply the loss which Art has suffered in your Julius, than the one who has followed with admiring sympathy his noble, constant, and successful strivings in these latter years, and who will ever bear his friendship faithfully in mind".
Reubke did not live long enough to build up a substantial body of work. However, his Piano Sonata (1857) is occasionally performed today, and his best known work, the Organ Sonata on the 94th Psalm (also 1857) is considered one of the finest 19th century organ pieces. Both works display the influence of Liszt, particularly his Piano Sonata.