After completing touring in 1996, Buckley started to write for a new album to be called My Sweetheart the Drunk. In 1997 he moved to Memphis, Tennessee, where he rented a shotgun house of which he was so fond he contacted the owner about the possibility of buying it. Buckley started recording demos on his own 4-track recorder. He went into the studio again, recruited a band, and plans for the new album looked hopeful.

On May 29, 1997, as the band's plane touched down on the runway to join him in his Memphis studio, Buckley went swimming in Wolf River Harbor, a tributary of the Mississippi River, while wearing steel-toed boots, all of his clothing, and singing along to a radio playing Led Zeppelin's "Whole Lotta Love". A roadie of Buckley's band, Keith Foti, remained ashore. After moving the radio and a guitar out of reach of the wake from a passing tugboat, Foti looked up to see that Buckley was gone. Despite a determined rescue effort that night, Buckley remained missing, and the search was called off the following day due to heavy rain. One week later his body was spotted by a tourist on a riverboat marina and was brought ashore.

The biography Dream Brother, written about him and his father, reveals that the night before his death Buckley reportedly admitted to several loved ones that he suffered from bipolar disorder. The autopsy confirmed that Buckley had taken no illegal drugs before his swim and a drug overdose was ruled out as the cause of death. He was 30 years old.

A recent statement from the Buckley estate insists:

Jeff Buckley's death was not "mysterious," related to drugs, alcohol, or suicide. We have a police report, a medical examiner's report, and an eye witness to prove that it was an accidental drowning, and that Mr. Buckley was in a good frame of mind prior to the accident.

After Buckley's death, a collection of demo recordings and a full length album he had been reworking for his second album were released as Sketches for My Sweetheart the Drunk - the compilation being overseen by Chris Cornell. Three other albums composed of live recordings have also been released, along with a live DVD of a performance in Chicago. A previously unreleased 1992 recording of "I Shall Be Released", sung by Buckley over the phone on live radio, was released on the album For New Orleans.